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FOREWORD 


The  U.S.  Army  Training  and  Doctrine  Command's  (TRADOC)  goals  include  increasing  training 
opportunities  for  all  soldiers,  improving  the  quality  of  instruction,  increasing  access  to  training, 
and  reducing  the  time  soldiers  spend  away  from  their  unit.  Accordingly,  there  is  intense  interest 
on  the  part  of  the  Army  to  consider  distance  learning  as  at  least  part  of  the  solution  toward  ad¬ 
dressing  these  goals.  Specifically,  the  Army  is  planning  to  convert  approximately  525  training 
courses  to  a  distance  learning  format.  This  future  reliance  on  distance  learning  to  deliver  train¬ 
ing  makes  it  incumbent  upon  the  Army  to  evaluate  the  effectiveness  of  this  training  method. 

This  study  originated  from  a  request  by  the  U.S.  Army  Sergeants  Major  Academy  (USASMA)  to 
examine  the  long  term  effectiveness  of  distance  learning.  The  Battle  Staff  NCO  Course 
(BSNCOC)  was  selected  due  to  its  importance  to  the  Army  and  its  relative  maturity  in  a  dis¬ 
tance  learning  format.  The  USASMA  requirement  stated  "As  USASMA  continues  to  rely  on  dis¬ 
tance  learning  techniques  to  design  and  develop  TATS  courses,  we  must  be  certain  the  methods 
we  choose  are  effective.  We  are  sacrificing  a  tried  and  true  method  that  works."  The  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  was  tasked  to  oversee  the  study,  which 
was  conducted  under  contract  to  Personnel  Decisions  Research  Institutes,  Inc. 


This  study  was  designed  to  determine  if  distance  learning,  in  comparison  to  residence  training, 
results  in  similar  levels  of  knowledge  retention  and  job  performance  in  the  areas  trained.  The 
results  of  this  study  were  briefed  to  TRADOC  at  the  Sergeants  Major  Academy  on  24  October 
2000.  The  findings  in  this  report  have  important  implications  for  the  efficacy  of  distance 
learning  methods  in  Army  Courses. 


jjfTA  M.  SIMUTIS 
echnical  Director 


EFFECTIVENESS  OF  DISTANCE  LEARNING  FOR  THE  BATTLE  STAFF  NCO  COURSE 
EXECUTIVE  SUMMARY _ _ _ 


Research  Requirement 


At  the  request  of  TRADOC,  a  field  study  was  conducted  to  evaluate  the  effectiveness  of  the  U.S. 
Army  Sergeants  Major  Academy's  Battle  Staff  NCO  Course  distance  learning  training  in  com¬ 
parison  with  training  conducted  in  residence.  Unlike  most  previoris  evaluations  of  distance 
learning  that  assessed  only  immediate  student  reactions  to  tire  course  or  their  end-of-course  test 
scores,  the  present  research  evaluated  course  graduates  after  they  were  on-the-job,  both  on  their 
course-relevant  job  knowledge,  as  measured  by  a  written  test,  and  on  their  job  performance,  as 
rated  by  their  supervisors. 


Procedure: 

To  create  a  basis  for  both  the  job  knowledge  and  performance  rating  scales,  those  Battle  Staff 
NCO  task  areas  that  are  targeted  by  the  BSNCO  course  were  identified.  Researchers  worked 
with  BSNCOC  instructors  to  identify  eight  task  dimensions  that  reflected  aU  important  job¬ 
relevant  areas  taught  in  the  course.  Written  knowledge  test  items  were  then  generated  for  each 
dimension.  A  pilot  study  was  conducted  with  NCOs  in  Bosnia  and  Germany  to  select  the  items 
in  the  final  instrument.  Rating  scales  were  also  developed  to  measure  job  performance  in  each 
of  the  eight  task  areas.  Task  labels  and  definitions  were  crafted,  along  with  behavioral  state¬ 
ments  to  anchor  ineffective,  mid-range,  and  effective  performance  levels  for  each  dimension; 
such  behavioral  anchors  help  raters  make  more  reliable  and  valid  job  performance  ratings.  A 
training  video,  with  introductory  remarks  by  the  Sergeant  Major  of  the  Army,  emphasized  the 
importance  of  participation  in  the  study  and  explained  to  the  supervisors  how  to  use  the  be¬ 
havioral  anchors  in  making  performance  ratings.  Finally,  a  work  experience  questionnaire  was 
developed  to  assess  how  much  opportunity  each  graduate  had  to  perform  in  each  task  area  on 
the  job. 

Criterion  data  were  collected  for  three  groups:  1)  NCOs  who  took  the  entire  Battle  Staff  Course 
in  residence  at  Fort  Bliss;  2)  NCOs  who  completed  a  pre-resident  self-study  phase  and  then  took 
a  shortened  course  via  distance  learning  technology;  and  3)  a  group  that  completed  a  pre¬ 
resident  self-study  phase  and  then  completed  a  compressed  course  in  residence  at  Fort  McCoy. 
Once  the  criterion  measures  were  finalized,  project  staff  traveled  to  Fort  Bragg  to  collect  data. 
Based  on  the  experience  there,  a  final  protocol  was  established  that  was  used  to  collect  data  in 
subsequent  locations  (Schofield  Barracks  and  Fort  SUl).  To  increase  the  sample  size,  a  mail-out 
version  of  the  data  collection  materials  was  developed.  A  total  of  427  packets  were  sent  to  Army 
installations  for  distribution  (67  of  these  were  returned  as  imdeUverable).  This  allowed  data 
collection  in  locations  where  it  was  not  feasible  to  have  project  staff  conduct  sessions.  The  com¬ 
bined  data  were  analyzed  to  determine  whether  there  were  differences  in  retained  job  knowl¬ 
edge  and  rated  job  performance  among  the  NCOs  who  had  taken  the  BSNCOC  through  the 
different  delivery  modalities. 
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Findings; 

Few  differences  were  found  in  the  job  performance  or  job  knowledge  levels  for  graduates  of  the 
different  versions  of  the  course.  That  is,  NCOs  performed  almost  equally  well  on  the  job  knowl¬ 
edge  test  and  received  similar  supervisory  ratings,  regardless  of  the  type  of  BSNCCXZ  they  at¬ 
tended.  We  conclude  that  the  distance  learning  version  of  the  course  results  in  subsequent  job 
performance  that  is  virtually  equivalent  to  that  of  NCOs  trained  via  a  residence  course. 


Utilization  of  Findings: 

These  results  indicate  that  distance  learning  technology  is  a  viable  alternative  to  residence  based 
training.  Such  findings  have  important  consequences  on  the  planned  conversion  of  several 
other  residential  coiirses  to  distance  learning. 
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BackgTovmd  of  Distance  Learning 

Distance  learning  can  be  defined  as  "structured  learning  that  takes  place  without  the  physical 
presence  of  the  instructor"  (Moore  &  Kearsley,  1996).  Contemporary  approaches  typically  take 
advantage  of  information  and  communications  technology  including  compressed  video, 
streaming  audio,  interactive  graphic  images,  and  teleconferencing  services.  Broadband  commu¬ 
nication  channels  enable  remote  locations  to  experience  full-motion  video  from  the  originating 
site. 

In  recent  years,  technology-based  instructional  formats,  including  distance  learning,  have  be¬ 
come  increasingly  popular.  For  example,  in  higher  education  in  1997,  more  than  50,000  courses 
were  taught  at  1,000  universities  to  more  than  7  million  students.  Also,  16%  of  corporate  train¬ 
ing  was  delivered  online  in  1997,  and  in  1998,  this  number  was  anticipated  to  be  28%  (Phillips, 
1998). 

Distance  learning  technology  can  greatly  reduce  travel  expenses,  enable  soldiers  to  stay  at  home 
with  their  families,  and  soldiers  in  tiie  reserve  component  can  keep  their  regular  jobs  while  still 
receiving  training  (Wisher  et  al.,  1999).  In  fact.  Wisher  and  Priest  (1998)  referred  to  an  example 
in  which  using  audio  teletraining  reduced  costs  by  more  than  $1000  per  person  for  a  three  week 
course.  The  Defense  Department  has  a  yearly  training  load  of  165,000  students  and  could  save  a 
substantial  amount  of  money  by  increasing  its  utilization  of  distance  learning  technology.  The 
Army's  future  reliance  on  distance  learning  to  deliver  training  makes  it  critical  for  the  Army  to 
evaluate  the  effectiveness  of  this  training  method. 

There  has  been  considerable  research  conducted  on  distance  learning,  mainly  in  civilian  educa¬ 
tion  settings  and  mostly  using  student  attitudes  or  end-of-course  learning  outcomes  as  criteria 
for  effectiveness.  Results  of  these  studies  vary  considerably.  Some  studies  find  satisfaction  with 
training  among  distance  learning  students  to  be  somewhat  lower  than  satisfaction  associated 
with  residence  training  (Maki  &  Maki,  2000).  In  contrast,  a  few  studies  report  that  distance 
learning  students  are  as  satisfied  as  traditional  classroom  students  (Chute,  Balthazar  &  Poston, 
1988;  Garrison,  1990;  Simpson,  Pugh,  &  Parchman,  1993).  Payne  (1999)  also  points  out  that  satis¬ 
faction  and  achievement  are  largely  independent.  Thus,  although  some  students  do  not  like  dis¬ 
tance  learning  as  well,  it  does  not  seem  to  prevent  them  from  effectively  learning  the  material. 

Regarding  learning  outcomes,  most  studies  have  foimd  that  distance  learning  students  perform 
as  weU  or  even  better  than  their  residence  course  cormterparts  on  end-of-training  knowledge 
tests  (Chute  et  al.,  1988;  Keene  &  Cary,  1990;  Payne,  1999;  Simpson,  Wetzel,  &  Pugh,  1995; 
Wisher,  Seidel,  Priest,  Knott,  &  Cumow,  1997).  For  instance,  Simpson  et  al.  fotmd  that  Navy  of¬ 
ficers  performed  equally  well  regardless  of  whether  their  course  was  conducted  via  a  traditional 
classroom  setting,  or  local  or  remote  video  teletraming  (VTT).  Flowever,  student  participation 
and  interaction  was  lower  in  the  two  VTT  classrooms. 

This  research  on  student  satisfaction  and  learning  during  the  course  is  important.  However, 
more  central  to  the  efficacy  of  distance  learning  in  the  Army  is  the  subsequent  on-the-job  per¬ 
formance  of  soldiers.  Does  distance  learning,  in  comparison  to  residence  training,  result  in  simi¬ 
lar  job  performance  in  the  areas  trained?  In  other  words,  the  effectiveness  of  a  training  program 
targeted  to  impart  specific  job  skills  is  best  evaluated  by  assessing  relevant  actual  job  perform- 
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ance  subsequent  to  the  training.  The  present  study  was  designed  to  evaluate  ihe  Battle  Staff 
NCO  Course  (BSNCOC)  in  just  this  way. 

Project  staff  administered  job  knowledge  tests  and  obtained  job  performance  evaluations  of  the 
NCOs  from  each  of  the  three  groups  (residence,  partial  residence,  and  VTT)  5  to  16  months  after 
graduation  from  the  BSNCOC.  Their  job  performance  and  scores  on  the  job  knowledge  test 
were  then  evaluated  to  assess  the  relative  effectiveness  of  the  three  versions  of  training  delivery. 

Method 


The  Battle  Staff  NCO  Course 


The  Battle  Staff  NCO  Coxirse  is  designed  to  train  battalion  and  brigade  staff  noncommissioned 
officers  (NCOs)  to  serve  as  integral  members  of  the  battle  staff  and  manage  daily  operations  of 
battalion  or  brigade  command  posts.  NCOs  learn  their  specific  staff  duties  and  become  familiar 
with  the  duties  of  other  staff  sections. 

Soldiers  with  the  "2S"  designation  assigned  to  a  Staff  NCO  position  in  sections  S1-S4  (SI  Per¬ 
sonnel  Section,  S2  Intelligence  Section,  S3  Operations  Section,  or  S4  Logistics  Section)  are  eligi¬ 
ble  to  attend.  The  version  of  the  course  that  a  soldier  attends  is  based  primarily  on  cost. 
Naturally,  if  a  VTT  site  is  available  where  the  soldier  is  stationed,  the  soldier  wiU  attend  this 
course.  Most  reserve  soldiers  attend  die  coiuse  at  Fort  McCoy  because  the  compressed  schedule 
of  working  through  the  weekends  allows  them  to  return  to  their  civilian  jobs  more  quickly.  Ac¬ 
tive  Component  soldiers  more  often  attend  the  course  at  Fort  Bliss,  although  they  may  attend 
any  of  the  three  versions  best  suiting  their  schedule.  The  versions  of  the  course  are  presented  in 
Table  1  and  described  in  detail  below. 

Table  1 

Battle  Staff  NCO  Course  Versions 


Fort  Bliss 


6  weeks  and  2  days  of  live  instruction  in  residence  at  Fort  Bliss,  TX 


Video  Teletraining  (VTT) 


Phase  1:  60  days  of  pre-resident,  paper-  Phase  2:  24  days  of  VTT  instruction  at 
based  self-study  remote  sites 


Fort  McCoy 


Phase  1:  60  days  of  pre-resident,  paper-  Phase  2:  21  days  of  live  instruction  in 

based  self-study  residence  at  Fort  McCoy,  Wl 


The  Residence  Course  at  Fort  Bliss 


The  former  course  at  Fort  Bliss,  Texas  was  six  weeks,  two  days  in  length  and  was  administered 
entirely  in  residence.  NCOs  in  this  course  worked  weekdays,  taking  the  weekends  off.  It  is  im¬ 
portant  to  note  that  this  version  of  the  course  has  now  been  changed  to  include  a  pre-resident 
phase,  and  the  actual  resident  portion  has  been  shortened  to  21  days.  Thus,  the  new  comrse  is 
quite  similar  to  the  Fort  McCoy  version  described  below. 
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The  Video  Teletraining:  (YTT)  Course 


This  version  had  two  phases.  During  Phase  1,  NCOs  studied  materials  at  home  for  60  days. 
Phase  2  lasted  24  academic  days  and  was  conducted  via  a  two-way  video  and  audio  connection, 
that  is,  in  a  distance  learning  format.  Delivery  was  through  a  compressed  signal  over  T-1  lines. 

Each  VTT  classroom  was  eqrdpped  with  several  cameras.  The  primary  camera  allowed  the  stu¬ 
dents  to  view  the  instructor  and  vice  versa.  It  was  soimd  sensitive  in  that  it  aimed  the  camera  at 
whomever  was  speaking  or  making  noise.  The  instructor/ operator  could  override  this  feature 
and  control  the  camera  manually  if  desired.  The  instructor  could  also  zoom  in  on  students  while 
they  completed  exercises.  The  second  camera  was  a  document  camera  mounted  on  an  adjust¬ 
able  arm.  The  instructor  could  use  it  to  display  training  materials  such  as  paper  products,  mod¬ 
els,  etc.  and  the  students  could  use  theirs  to  show  their  work  to  other  students  at  remote 
locations  or  the  instructor.  A  third  camera  projected  computer  graphics,  Internet  sites,  and  CD- 
ROMs.  A  video  cassette  recorder  was  also  available  for  viewing  video  tapes. 

The  Course  at  Fort  McCoy 

A  comparable  course,  with  a  smaller  student  load,  was  conducted  at  the  USARTC,  Fort  McCoy, 
Wisconsin.  This  version  of  the  course  also  had  two  phases.  like  the  VTT  version.  Phase  1  con¬ 
sisted  of  a  60  day,  self-study  program.  Phase  2  was  21  consecutive  days  spent  in  residence  at 
Fort  McCoy.  This  shortened  course  was  designed  to  better  accommodate  soldiers  in  tibe  Guard 
and  Reserve  Components  of  the  Army,  although  Regular  Army  soldiers  were  allowed  to  attend 
as  well.  All  three  training  formats  covered  the  same  content. 

Development  of  the  Performance  Measmes 

PDRI  developed  two  job  performance  measures,  a  job  knowledge  test,  and  performance  rating 
scales.  To  accomplish  this,  we  first  worked  with  BSNCOC  instructors  to  develop  a  task  list  that 
represented  all  of  the  important  tasks  taught  in  the  course.  Eight  tasks  resulted  (e.g..  Assist  in 
Planning  of  Army  Operations,  Assist  in  the  Military  Decision  Making  Process;  see  Table  2,  for 
the  complete  list,  including  task  area  definitions). 

Table  2 

Task  Dimension  List  with  Definitions 


Dimension  1  —  Assist  in  the  Military  Decision-Making  Process  (MDMP) 

Understands  and  participates  in  MDMP.  Responds  to  requests  for  information  regarding  the  MDMP. 

Dimension  2  —  Prepare  Combat  Orders  or  Annexes  (POA) 

Is  knowledgeable  about  the  5-paragraph  order  format  and  the  different  variations  of  orders. 

Dimension  3  —  Prepare  or  Construct  Graphics  or  Overlays  (GO) 

Uses  graphics  and  overlays  to  convey  operations  orders.  Understands  and  can  produce  military  symbols  and  graphics. 
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Table  2  (continued) 


Dimension  4  —  Understanding  Intelligence  Preparation  of  the  Battlefield  (IPB) 

Understands  and  participates  in  the  wargaming  process  and  identifies  courses  of  action  based  on  IPB  process  and  products. 
Gathers  and  uses  available  IPB  products  and  resources. 


Dimension  5  —  Assist  in  Planning  of  Army  Operations  (AO) 

Understands  Army  doctrine  and  military  operations  including:  the  nine  principles  of  war,  tenets  of  combat  operations:  Battle¬ 
field  Operating  Systems  (BOS);  participates  in  the  planning  process. 

Dimension  6  —  Assist  in  the  Planning  and  Execution  of  Combat  Support  (CS)  and  Combat  Service  Support  (CSS) 
Orchestrates  CS  and  CSS  efforts;  understands  the  role,  function,  and  structure  of  CS  and  CSS  in  support  of  Army  operations. 


Dimension  7  —  Manages  Recordkeeping  (RR) 

Keeps  records  on  staff  journal  log,  DA  Form  1594,  etc,  in  accordance  with  unit  Standard  Operating  Procedure  (SOP);  main¬ 
tains  the  daily  staff  journal. 

Dimension  8  —  Prepare  and  Conduct  Military  Briefings  (MB) 

Prepares  and  conducts  military  briefings;  delivers  briefings;  understands  different  types  of  briefings  and  when  to  apply  them. 


Tob  Knowledge  Test  Development 

First,  with  the  help  of  a  BSNCOC  instructor,  we  wrote  knowledge  items  for  each  of  the  task  ar¬ 
eas,  approximately  8-9  items  per  task,  resulting  in  70  multiple-choice  items.  The  70  items  were 
split  into  two  forms  for  a  pilot  test  to  avoid  burdening  the  soldiers  with  an  overly  lengthy  exam. 
A  total  of  62  BSNCOC  students  near  graduation  in  Germany  and  Bosnia  completed  one  of  the 
two  versions. 

The  psychometric  properties  of  these  70  items,  including  the  difficulty,  item-total  correlation, 
and  coefficient  alpha  without  the  item  were  evaluated.  Questions  that  were  overly  easy  or  dif¬ 
ficult  were  dropped  because  they  do  not  distinguish  very  weU  among  people.  That  is,  if  almost 
everyone  answers  a  question  correctly  or  incorrectly,  it  does  not  help  differentiate  the  amount 
of  knowledge  different  people  retained. 

PDRI  staff  also  examined  the  item-total  correlations  for  each  of  the  test  items.  Items  that  did  not 
correlate  well  with  the  rest  of  the  test  were  also  candidates  for  deletion.  Another  statistic  we 
considered  was  the  coefficient  alpha  if  the  item  was  deleted.  Coefficient  alpha  is  a  measure  of 
internal  reliability  or  consistency  that  indicates  how  well  the  items  of  a  scale  relate  to  each  other. 
If  an  item  was  unrelated  to  the  rest  of  the  test  items,  then  the  coefficient  alpha  is  improved  by 
deleting  such  items.  The  overall  coefficient  alphas  for  each  of  the  tests  were  .43  for  Form  A  and 
.50  for  Form  B. 
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Final  decisions  concerning  which  items  to  keep  were  based  on  the  psychometric  properties  de¬ 
scribed  above  and  balancing  the  number  of  items  for  each  dimension.  This  resulted  in  the  42 
items  for  the  final  test,  shown  in  Appendix  A.  To  control  for  order  effects,  two  forms  of  the  test 
were  created  consisting  of  the  same  questions  in  different  orders. 

Sources  of  Tob  Performance  Ratings 

The  three  somces  most  often  used  to  rate  job  performance  are  self,  peers,  and  supervisors.  Logi¬ 
cally,  self-ratings  should  contain  a  great  deal  of  information  (Borman,  1991).  They  also  distin¬ 
guish  among  performance  dimensions  better  than  peer  and  supervisor  ratings  (i.e.,  less  halo 
error;  Heneman,  1974;  Kirchner,  1965). 

Unfortunately,  self-ratings  tend  to  be  more  lenient  than  supervisor  or  peer  ratings  (Kirchner, 
1965;  Parker,  Taylor,  Barrett,  &  Martens,  1959).  In  their  meta-analysis,  Harris  and  Schaubroeck 
(1988)  found  that  self-ratings  were  more  lenient  than  supervisor  ratings  by  over  half  a  standard 
deviation  and  were  more  lenient  than  peer  ratings  by  a  quarter  of  a  standard  deviation.  This 
leniency  becomes  even  more  problematic  if  some  self-raters  are  more  lenient  than  others  (Bor¬ 
man,  1991).  Self-ratings  also  correlate  lower  with  other  raters'  ratings  (.22),  compared  to  peer- 
supervisor  rating  correlations  (.34,  Conway  &  Huffcutt,  1997). 

Peer  raters  can  be  a  viable  alternative  to  self-ratings  because  peers  should  have  a  great  deal  of 
information  about  a  soldier's  typical  performance  without  as  much  of  a  leniency  problem. 
However,  peers  often  lack  experience  making  performance  ratings  whereas  supervisors  usually 
have  better  calibrated  performance  norms  because  of  their  experience  in  evaluating  subordi¬ 
nates  (Borman,  1991). 

Although  supervisors  may  not  have  access  to  as  much  information  as  self-raters  and  peers,  su¬ 
pervisors  are  likely  to  pay  more  careful  attention  to  performance  because  it  is  part  of  their  job  to 
do  so.  Others  may  be  less  Hkely  to  observe  performance  examples,  properly  encode  them,  or 
retain  and  recall  performance  in  a  manner  suitable  for  evaluation  (AUen,  Bcimard,  Rush,  &  Rus¬ 
sell,  1996). 

Perhaps  the  most  compelling  argument  for  using  supervisory  ratings  is  that  they  are  more  reli¬ 
able  (KHeger  &  Mosel,  1953;  Pulakos  &  Borman,  1988;  Springer,  1953).  Viswesvaren,  Ones  and 
Schmidt  (1996)  foimd  in  their  meta-analysis  that  supervisor  ratings  had  higher  interrater  reli¬ 
ability  (.52)  than  did  peer  ratings  (.42).  Similarly,  Conway  and  Huffcutt  (1997)  found  supervi¬ 
sory  ratings  to  be  more  reliable  (.50)  tiian  both  peer  (.37)  and  subordinate  ratings  (.30). 
Reliability  is  critical  to  performance  ratings  because  it  puts  an  upper  limit  on  validity.  There¬ 
fore,  supervisors  were  deemed  the  most  appropriate  source  to  make  job  performance  ratings  for 
this  study. 

Tob  Performance  Scale  Development 

The  BSNCOC  instructors  worked  with  PDRI  to  create  supervisory  performance  rating  scales 
targeted  toward  the  eight  task  areas.  For  each  task  area,  we  generated  a  definition  and  tiien 
prepared  behavioral  statements  to  anchor  the  ineffective  (i.e.,  1-2),  mid-range  (i.e.,  3-5),  and  ef¬ 
fective  (i.e.,  6-7)  performance  levels.  These  behavioral  statements  were  intended  to  provide 
benchmarks  for  supervisor  raters  to  use  when  evaluating  their  subordinates.  Research  (e.g.. 
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Borman  &  Pulakos,  1986)  suggests  that  these  anchors  help  raters  to  generate  more  reliable  rat¬ 
ings.  Thus,  the  PDRI  staff  wrote  draft  behavioral  statements  after  oxir  meetings  with  the  instruc¬ 
tors,  obtained  feedback  from  this  group,  and  made  revisions  based  on  the  feedback.  The  final 
behavior-based  scales  appear  in  Appendix  B. 

In  addition,  our  staff  prepared  a  rater  training  videotape  to  help  supervisor  raters  understand 
the  importance  of  the  study  and  to  make  more  reliable  and  accurate  ratings  of  job  performance. 
The  video  began  with  a  clip  of  the  Sergeant  Major  of  fire  Army  encouraging  the  viewer  to  par¬ 
ticipate  conscientiously  in  the  research.  The  rest  of  the  9  minute  video:  (1)  explained  the  features 
of  file  behavior-based  rating  scales,  especially  the  proper  use  of  the  behavioral  anchors;  (2)  de¬ 
scribed  common  rating  errors  (e.g.,  halo  and  leniency)  and  how  to  avoid  them;  and  (3)  encour¬ 
aged  supervisors  to  be  as  accurate  as  possible  in  their  for-research-only  performance  ratings. 

Sample 

Performance  data  were  gathered  by  PDRI  staff  at  Fort  Bragg,  Fort  Sill,  and  Schofield  Barracks. 
To  further  increase  the  sample  size,  we  also  conducted  a  mail-out  data  collection  at  11  sites, 
sending  a  total  427  packets  with  67  returned  as  imdeUverable.  For  file  mail  version  of  the  study, 
the  responses  were  mainly  from  Germany,  Forts  Hood,  Polk,  Campbell,  and  Carson.  Response 
rates  from  aU  sites  are  displayed  in  Table  3.  The  residence  graduate  group  from  in-person  and 
mail-out  administration  of  the  performance  measures  consisted  of  92  NCOs  who  took  the  job 
knowledge  test  and  80  who  were  rated  by  their  supervisor.  The  VTT  group  consisted  of  57 
NCOs  with  test  scores  and  47  with  supervisory  ratings.  Finally,  23  NCOs  from  Fort  McCoy 
completed  job  knowledge  tests  and  18  supervisors  rated  their  performance. 

Table  3 
Sample  Totals 


Job  Knowledge  Test 


Site 

Bliss 

vn 

McCoy 

Total 

Fort  Bragg 

56 

12 

0 

68 

Fort  Sill 

2 

12 

0 

14 

Schofield  Bks 

15 

0 

17 

32 

Mail  Version 

19 

33 

6 

58 

Total 

92 

57 

23 

172 

Supervisor  Ratings 


Site 

Bliss 

VTT 

McCoy 

Total 

Fort  Bragg 

48 

11 

0 

59 

Fort  Sill 

3 

12 

0 

15 

Schofield  Bks 

9 

0 

12 

21 

Mail  Version 

20 

24 

6 

50 

Total 

80 

47 

18 

145 
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Table  3  (continued) 


Job  Knowledge  Test:  Mail  Version  by  Location 

Site 

Bliss 

vn 

McCoy 

Total 

Fort  Benning 

0 

1 

0 

1 

Camp  Blanding 

0 

0 

0 

0 

Fort  Campbell 

7 

1 

0 

8 

Fort  Carson 

4 

0 

2 

6 

Germany 

0 

17 

0 

17 

Fort  Hood 

5 

4 

0 

9 

Fort  Knox 

0 

0 

4 

4 

Fort  Leonard  Wood 

0 

0 

0 

0 

Fort  Polk 

0 

9 

0 

9 

Fort  Riley 

0 

1 

0 

1 

Fort  Stewart 

3 

0 

0 

3 

Total 

19 

33 

6 

58 

Supervisor  Ratings:  Mail  Version  by  Location 


Site 

Bliss 

vn 

McCoy 

Total 

Fort  Benning 

0 

0 

0 

0 

Camp  Blanding 

0 

0 

0 

0 

Fort  Campbell 

6 

3 

0 

9 

Fort  Carson 

5 

0 

1 

6 

Germany 

0 

9 

1 

10 

Fort  Hood 

6 

6 

0 

12 

Fort  Knox 

0 

0 

4 

4 

Fort  Leonard  Wood 

0 

0 

0 

0 

Fort  Polk 

0 

6 

0 

6 

Fort  Riley 

0 

0 

0 

0 

Fort  Stewart 

3 

0 

3 

Total 

20 

24 

6 

50 

The  soldiers  had  graduated  from  the  BSNCOC  4.75  to  16  months  before  the  job  performance 
measures  were  administered,  with  an  average  of  9.62  months  since  graduation.  This  window  of 
time  was  considered  long  enough  for  soldiers  to  have  had  the  opportunity  to  work  in  the 
BSNCOC  task  areas  but  not  so  long  that  the  class  materials  would  be  forgotten.  There  were  es¬ 
sentially  no  differences  in  the  number  of  months  since  graduation  among  NCOs  from  the  resi¬ 
dence  course  at  Fort  Bliss  (M=9.55  months),  the  partial  residence  group  at  Fort  McCoy  (M=9.48 
months),  and  the  VTT  course  (M=9.79  months). 

All  members  of  the  sample  were  in  the  Active  Army  and  most  were  in  paygrade  E-6  (42%)  or 
E-7  (53%,  see  Table  4).  hi  terms  of  education  level,  most  (65%)  had  completed  some  coUege,  and 
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some  (11%)  had  finished  their  degrees.  Overall,  the  average  graduate  was  34.4  years  old  with 
13.74  years  in  service.  Finally,  the  Fort  Bliss  graduates  were  quite  evenly  divided  across  the 
three  service  branches,  with  about  a  third  of  the  NCOs  from  each  area.  The  VTT  group  had  sub¬ 
stantially  fewer  NCOs  in  the  Combat  Support  branch  (17%)  compared  to  the  other  two  branches 
with  44%  in  Combat  Arms  and  39%  in  Combat  Service  Support. 


Table  4 

Description  of  BSNCOC  Sample 


Bliss 

VTT 

McCoy 

Overall 

Months  Since  Graduation 

Mean 

9.55 

9.79 

9.48 

9.63 

SD 

2.80 

2.64 

2.54 

2.74 

N 

109 

58 

27 

167 

Range 

4.75-16 

6-13.5 

6-15 

4.75-16 

Age 

Mean 

34.14 

35 

38.43 

34.40 

SD 

4.86 

4.85 

5.62 

4.65 

N 

111 

60 

7 

178 

Range 

23-50 

Time  in  Service  (Years) 

Mean 

13.67 

14.25 

16 

13.74 

SD 

3.23 

3.90 

4 

3.58 

N 

111 

60 

7 

178 

Range 

5-24 

Paygrade 

E-6 

46% 

36% 

29% 

42% 

E-7 

48% 

59% 

71% 

53% 

E-8 

5% 

5% 

0% 

5% 

Number  Missing 

0 

9 

20 

29 

N 

110 

68 

27 

205 

Educational  Level 

High  School 

6% 

6% 

14% 

7% 

Some  College 

63% 

68% 

71% 

65% 

Associate’s  Degree 

19% 

17% 

0% 

18% 

College  Graduate 

10% 

7% 

14% 

9% 

Graduate  Degree 

2% 

2% 

0% 

2% 

Number  Missing 

0 

9 

20 

29 

N 

110 

68 

27 

205 
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Table  4  (continued) 


Bliss 

VTT 

McCoy 

Overall 

Branch 

Combat  Arms 

35% 

44% 

29% 

37% 

Combat  Support 

35% 

17% 

14% 

28% 

Combat  Service  Support 

31% 

39% 

57% 

35% 

Number  Missing 

0 

9 

20 

29 

N 

110 

68 

27 

205 

Note:  N/A  indicates  not  available. 


Administration 

PDRI  staff,  along  with  a  representative  from  the  Sergeants  Major  Academy,  traveled  to  Fort 
Bragg,  Fort  SiU,  and  Schofield  Barracks  to  collect  data.  The  staff  explained  to  the  soldiers  that 
this  study  was  for  research  purposes  only  and  was  intended  to  analyze  the  effectiveness  of  the 
BSNCOC.  All  soldiers  were  assured  that  their  responses  and  scores  would  be  kept  confidential. 
The  NCOS  then  viewed  a  video  of  the  Sergeant  Major  of  the  Army  encouraging  participation. 
Next,  the  soldiers  were  asked  to  take  the  job  knowledge  test  without  using  any  references.  They 
were  encouraged  to  try  their  best  and  to  guess  if  they  were  unsure  of  an  answer.  The  NCOs 
then  completed  a  self  experience  rating  form  indicating  the  amoimt  of  experience  they  had  in 
each  of  the  eight  performance  areas.  Finally,  they  were  given  an  opportunity  to  provide  feed¬ 
back  on  a  comments  sheet. 

Then  the  performance  rating  scales  were  administered  to  supervisor  groups  to  evaluate  the  job 
performance  of  their  recent  BSNCOC  graduates.  They  too  were  assured  of  the  confidentiality  of 
their  responses  and  that  the  study  was  for  research  proposes  only.  After  viewing  the  rater  train¬ 
ing  videotape,  supervisors  rated  their  subordinate  NCOs  on  the  eight  performance  dimensions 
described  earlier.  The  raters  could  choose  to  not  make  a  rating  if  a  particular  dimension  was  not 
part  of  the  subordinate  NCO's  job;  an  NPJ  (Not  Part  of  Job)  option  was  provided  for  that  possi¬ 
bility.  Finally,  the  rating  directions  were  also  in  written  form  on  the  performance  rating  scales 
boolklet,  and  raters  indicated  on  an  experience  rating  sheet  the  length  of  time  they  had  known 
the  NCO  and  amoimt  of  experience  the  soldier  had  in  each  of  the  eight  task  areas.  The  supervi¬ 
sors  also  had  the  opportunity  to  complete  a  comments  sheet.  The  Dimension  Definitions,  Per¬ 
formance  Rating  Scales,  the  Performance  Rating  Sheet,  and  the  Experience  Rating  Form  can  be 
found  in  Appendix  B.  For  the  maU-out  administration,  aU  of  the  above  forms  were  sent,  along 
with  a  cover  sheet  explaining  tixe  importance  of  the  study  and  a  detailed  directions  sheet. 
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Results 


Tob  Knowledge  Test 

An  analysis  of  covariance  (ANCOVA)  was  used  to  analyze  the  results,  with  the  NCO's  self  ex¬ 
perience  ratings  for  each  dimension  used  as  the  covariate.  Experience  ratings  were  considered 
appropriate  covariates  in  that  graduates  with  less  job  experience  in  a  task  area  essentially  had 
their  job  knowledge  test  scores  adjusted  higher  to  allow  a  fairer  comparison  of  NCOs  with  dif¬ 
fering  opportunities  to  perform  in  that  area. 

On  average,  the  test  results  for  the  three  groups  were  almost  identical.  The  Fort  BHss  group  had 
an  overall  average  of  61%  correct,  the  VTT  group  62%,  and  the  McCoy  group  63%,  (F=.57, 
p=.57).  Table  5  displays  the  means  and  standard  deviations;  tire  ANCOVA  results  by  dimension 
can  be  found  in  Appendix  C.  Only  two  significant  between-group  differences  appeared.  On  the 
knowledge  test  for  Combat  Support/  Combat  Services  Support  (CSS),  there  was  an  overall  sig¬ 
nificant  difference  among  the  three  groups  (F=5.014,  p=.008).  This  analysis  revealed  that  the  Fort 
Bliss  group  (M=58.15%,  SD=.21)  scored  significantly  higher  than  the  Fort  McCoy  group 
(M=42.39%,  SD=.25)  on  the  CSS  dimension  (f=3.08,  p=.003). 


Table  5 

Job  Knowledge  Test  Results 


Bliss 

VTT 

McCoy 

Overail 

Recordkeeping 

Mean 

84% 

84% 

86% 

84% 

SD 

0.17 

0.20 

0.15 

0.18 

Briefings 

Mean 

75% 

68% 

74% 

72% 

SD 

0.24 

0.22 

0.27 

0.24 

Combat  Orders 

Mean 

68% 

71% 

78% 

70% 

SD 

0.20 

0.20 

0.15 

0.19 

Planning 

Mean 

59% 

68% 

65% 

63% 

SD 

0.26 

0.26 

0.21 

0.25 

Intelligence  Preparation 

Mean 

59% 

66% 

61% 

62% 

SD 

0.20 

0.23 

0.27 

0.22 

Combat  Support/CSS 

Mean 

58% 

51% 

42% 

54% 

SD 

0.21 

0.25 

0.25 

0.24 

Graphics/Overlays 

Mean 

55% 

55% 

58% 

56% 

SD 

0.21 

0.23 

0.20 

0.21 
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Table  5  (continued) 


Bliss 

vn 

McCoy 

Overall 

Assist  in  the  Military  Decision-Making  Process  (MDMP) 

Mean 

47% 

50% 

50% 

49% 

SD 

0.18 

0.22 

0.17 

0.20 

Overall 

Mean 

61% 

62% 

63% 

62% 

SD 

0.11 

0.15 

0.10 

0.12 

N 

89 

55 

23 

167 

When  the  Fort  McCoy  group  was  not  considered,  t-test  results  revealed  an  additional  significant 
difference  in  knowledge  test  performance  for  the  dimension  Assists  in  Planning  Army  Opera¬ 
tions  (AO).  For  the  AO  dimension,  the  VTT  group  (M=68.0%,  SD=.26)  outperformed  the  Fort 
Bliss  group  (M=59.33%,  SD=.26),  with  f=-2.075,  p=.040.  Additionally,  the  number  of  months 
since  the  soldiers'  graduation  from  the  NCO  course  did  not  affect  their  test  score  performance 
iF=  09,  p=  77). 


Tob  Performance  Ratings 

The  distribution  of  the  ratings  can  be  found  in  Figure  1.  Notice  that  the  ratings  are  approxi¬ 
mately  normally  distributed  with  a  slight  negative  skew,  as  is  common  in  supervisory  ratings. 
This  indicates  that  the  ratings  did  not  contain  much  error  due  to  leniency. 


Figure  1.  Distribution  of  supervisory  ratings 
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The  ratings  were  highly  similar  across  the  three  groups,  with  a  mean  of  5.1  for  both  the  Fort 
Bliss  and  VTT  group  and  4.7  for  the  Fort  McCoy  group  on  a  1  =  very  ineffective  to  7  =  very  ef¬ 
fective  scale.  These  differences  are  not  significant  (F=1.6,  p=.21).  Furtiier,  only  one  significant 
difference  appeared  in  the  analysis  of  the  dimensional  supervisory  performance  ratings.  Man¬ 
ages  Recordkeeping  (RR)  had  a  significant  overaU  F=3.86,  p=.029.  Post  hoc  t-tests  revealed  that 
the  VTT  group  (M=5.89,  SD=1.16)  was  rated  marginally  higher  by  their  supervisors  on  Manages 
Recordkeeping  than  was  the  McCoy  group  (M=5.00,  SD=1.35),  with  f=1.96,  p=.055.  A  complete 
table  of  ratings  is  provided  below  (Table  6)  and  Appendix  D  displays  results  from  the  ANCO- 
VAs. 


Table  6. 

Supervisory  Ratings  by  Task  Dimension  (Rank  ordered  -  7-point  scale) 


Bliss 

vn 

McCoy 

Overall 

Recordkeeping 

Mean 

5.61 

5.89 

5.00 

5.64 

SD 

1.24 

1.16 

1.35 

1.24 

Briefings 

Mean 

5.29 

5.28 

5.00 

5.25 

SD 

1.23 

1.14 

1.57 

1.25 

Combat  Orders 

Mean 

5.20 

4.97 

4.64 

5.06 

SD 

0.99 

1.55 

1.69 

1.30 

Planning 

Mean 

5.09 

4.86 

4.31 

4.92 

SD 

1.04 

1.42 

1.18 

1.21 

Intell.  Prep. 

Mean 

4.46 

4.59 

4.23 

4.47 

SD 

1.17 

1.46 

1.09 

1.25 

Combat  Support/CSS 

Mean 

4.87 

5.11 

4.54 

4.89 

SD 

1.01 

1.55 

1.45 

1.26 

Graphics/overlays 

Mean 

5.51 

5.42 

5.57 

5.49 

SD 

1.22 

1.44 

1.28 

1.29 

MDMP 

Mean 

4.84 

4.88 

4.50 

4.82 

SD 

1.22 

1.01 

1.31 

1.16 

Overall 

Mean 

5.10 

5.10 

4.74 

5.07 

SD 

0.74 

1.06 

0.89 

0.87 

N 

61 

37 

14 

113 

Note:  These  analyses  included  only  NCOs  who  completed  the  job  knowledge  test  and  had  supervisory  performance  ratings, 
resulting  in  a  smaller  N  than  those  listed  on  Table  3. 


Because  we  are  exploring  whether  or  not  differences  among  the  groups  exist,  the  effect  sizes 
(eta-squared  in  Appendix  D)  are  important.  These  effect  sizes  are  quite  small  (median  eta- 
squared  =  .014),  reinforcing  the  finding  that  job  performance  is  very  similar  across  the  three 
groups. 

Overview  of  the  Written  Comments 


Overall,  both  the  supervisors  and  NCOs  indicated  that  this  course  is  highly  important  to  their 
training  and  that  the  material  they  learned  was  useful.  The  comments  regarding  the  Fort  BHss 
residence  training  were  quite  positive.  Soldiers  appreciated  the  group  interaction  at  Fort  Bhss 
and  felt  they  could  focus  more  effectively  because  they  were  away  from  their  home  base.  Some 
supervisors  even  conceded  that  if  the  soldiers  were  not  away  from  their  home  base,  they  would 
ask  their  NCOs  to  perform  some  of  their  regular  duties  while  attending  the  course. 

The  comments  regarding  the  VTT  and  Fort  McCoy  courses  were  more  mixed.  Although  three 
soldiers  commented  that  they  received  good  instruction  and  Hked  the  VTT  format,  nine  be¬ 
lieved  it  was  more  difficult  to  learn  in  that  setting.  They  stated  that  they  would  have  preferred 
more  one-on-one  instruction  and  cited  problems  with  the  pre-resident  portion  of  the  course.  For 
example,  one  soldier  believed  the  pre-resident  portion  was  unnecessary  and  admitted  to  skip¬ 
ping  that  portion  due  to  other  pressing  obligations.  However,  three  soldiers  believed  it  was  vi¬ 
tal,  and  they  wanted  more  time  and  assistance  completing  the  exercises.  Regarding  Fort  McCoy, 
one  NCO  commented  that  the  course  worked  weU,  yet  three  soldiers  thought  it  was  too  short. 
The  soldiers  were  evenly  divided  in  their  views  about  whether  or  not  they  should  work  through 
the  weekends,  with  some  in  favor  of  it  to  save  time  and  money  and  others  opposing  it  because 
they  desired  a  break.  Appendices  E  and  F  contain  the  verbatim  written  comments  from  the  su¬ 
pervisors  and  NCOs. 


Conclusions 

This  study  was  designed  to  evaluate  the  relative  effectiveness  of  the  residence  and  distance 
learning  versions  of  the  BSNCOC  with  respect  to  subsequent  job  performance  levels  of  the 
graduates.  A  job  knowledge  test  was  intended  to  assess  knowledge  retention  in  the  task  areas 
taught  in  the  course.  The  performance  rating  scales  required  supervisors  to  evaluate  the  job  per¬ 
formance  of  the  BSNCOC  graduates  in  the  same  task  areas.  Results  suggested  almost  no  be- 
tween-group  differences  on  either  performance  measure.  Knowledge  retention  relative  to 
BSNCOC  tasks  and  rated  job  performance  in  these  task  areas  were  almost  exactly  the  same  for 
resident  and  distance  learning  graduates. 

These  results  have  important  implications  for  the  planned  conversion  of  many  residence 
courses  to  a  distance  learning  format.  They  provide  evidence  supporting  the  efficacy  of  tihe  VTT 
delivery  method  for  military  training  when  the  target  is  enhanced  on-the-job  performance. 

Thus,  results  of  this  study  directly  address  the  request  made  by  the  U.S.  Army  Sergeants  Major 
Academy  (USASMA)  that  the  U.S.  Army  Research  Institute  (ARI)  evaluate  the  long-term  effec¬ 
tiveness  of  distance  learning.  Based  on  the  findings  presented  here,  it  appears  that  distance 
learning  delivery  of  Army  courses  will  result  in  no  decrements  to  knowledge  retention  or  sub¬ 
sequent  job  performance  compared  to  residence  course  delivery  of  training. 
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Appendix  A  -  Final  Knowledge  Test  Items 


A-1 


Instructions 


We  are  asking  you  to  take  this  test  as  part  of  an  important.  Sergeant  Major  of  the  Army 
sponsored  research  project  to  evaluate  NCOs  retention  of  Battle  Staff  Course  material.  Please  try 
your  best  to  airswer  the  questions  correctly.  Your  responses  are  anonymous  and  will  be  used 
for  research  purposes  only.  Please  complete  this  test  alone  and  without  using  any  reference 
materials. 

For  each  of  the  42  questions,  indicate  your  answer  by  marking  an  X  next  to  the  response  option 
you  believe  is  correct.  If  you  do  not  know  the  answer  to  a  question,  you  may  guess.  There  is 
no  penalty  for  guessing.  The  test  is  not  timed  so  you  may  take  as  long  as  you  like. 

Thank  you  for  helping  with  this  important  research  project. 
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1.  The  role  of  doctrine  is: 


_ a.  Defined  in  the  Ranger  Handbook,  (SH  21-75) 

_ b.  A  statement  of  how  America's  Army  fights 

_ c.  Strategy  as  outlined  in  the  Monroe  Doctrine 

_ d.  Both  a  and  b  are  correct 

2.  The  sequence  of  the  five  (5)  paragraph  OPORD  is: 

_ a.  Situation,  mission,  execution,  service  and  support,  risk 

assessment 

_ b.  Mission,  situation,  service  and  support,  risk  assessment, 

execution 

_ c.  Execution,  mission,  situation,  command  and  signal,  service  and 

support 

_ d.  Situation,  mission,  execution,  service  and  support,  command 

and  signal 

3.  US  Army  and  allied  ground  forces  are  depicted  on  an  overlay  in  the  color  of: 

_ a.  Green 

_ b.  Yellow 

_ c.  Red 

_ d.  None  of  the  above 

4.  The  principles  of  war  first  introduced  in  1921,  although  slightly  revised,  contain 
how  many  principles  today? 

5 

0  a.  11 

3  - 

_ b.  7 

_ c.  9 

_ d.  6 
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5.  The  acronym  WSRO  refers  to: 

_ a.  Wanted  supplies  request  order 

_ b.  Weapons  system  replacement  operations 

_ c.  Weather  system  radar  operator 

_ d.  Weapon  system  request  order 

6.  Standardized  unit  labeling  field  (Foxtrot)  refers  to: 

_ a.  The  size  of  a  unit 

_ b.  A  unit  that  has  been  reinforced  or  detached 

_ c.  The  parent  headquarters  of  the  unit 

_ d.  Team  or  task  force  designation 

7.  The  Field  Manual  used  for  US.  Army  and  USMC  land-based  warfighting 
symbology  is: 

_ a.  FM 101-5 

_ b.  FMlOl-5-1,  MCRP  5-2A 

_ c.  Military  Standard  2525A 

_ d.  FM  22-5,  MCRP  21-20C 

8.  The  acronym  MCOO  is  defined  as: 

_ a.  Military  combination  overlay  overview 

_ b.  Modified  combination  overlay  overview 

3 

_ c.  Modified  combined  obstacle  overlay 

_ d.  Military  combined  obstacle  overlay 
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^  10.  Intelligence  Preparation  of  the  Battlefield  (IPB)  consists  of  how  many  steps? 

B 

0  a.  4 

4 

0  b.  6 

1 

_ c.  7 

_ d.  5 


^  11.  The  types  of  military  briefing  include: 

B 

0 

8  _ a.  Operational,  logistical,  informational 

0 

1  _ b.  Staff,  information,  operations,  decision 

_ c.  Personnel  statistics,  incident,  Scifety,  coordination 

_ d.  Decision,  tactical,  staff,  information 


^  12.  Coordinating  responsibility  for  war-gaming  belongs  to: 

M 

p  _ a.  S-3/G-3  Operations  Officer 

0 

1  _ b.  S-2/G-2  Security  Officer 

0 

^  _ c.  CofS/XO  Executive  Officer 

_ d.  CSM/Operations  NCO 
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1 

0 

3 


p 

o 

A 

0 

2 

0 

4 


13.  Select  the  correct  response: 

_ a.  MDMP  is  the  Army's  single,  established,  and  proven  analytical 

process  and  an  adaptation  of  the  Army's  analytical  approach  to 
problem  solving. 

_ b.  The  Commander  manages,  coordinates,  and  disciplines  the 

staff's  work  and  provides  quality  control. 

_ c.  Step#l  of  the  MDMP  is  Course  of  Action  (CO A)  development. 

_ d.  MDMP  is  a  process  which  applies  only  to  Intelligence 

Preparation  of  the  Battlefield,  (IPB). 

14.  The  Military  Decision  Making  Model  consists  of: 

_ a.  A  five  (5)  paragraph  format 

_ b.  A  seven  (7)  step  process 

_ c.  Decisions  based  on  risk  management  only 

_ d.  Decisions  made  only  by  Staff  Officers 

15.  The  first  step  of  Mission  Analysis  is: 

_ a.  Write  the  restated  mission 

_ b.  Conduct  IPB 

_ c.  Analyze  the  higher  headquarters  order 

_ d.  Record  staff  estimates  cind  input 

16.  Classification  marking  and  procedures: 

_ a.  Are  outlined  in  AR380-5 

_ b.  Are  placed  at  the  bottom  of  each  page 

_ c.  Only  the  original  signed  copy  needs  classification  markings 

_ d.  Both  b  and  c  are  correct 
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17.  Additions  to  an  OPORD  that  you  may  use  are: 

_ a.  An  event  matrix 

_ b.  A  graphic  overlay 

_ c.  An  annex 

_ d.  All  of  the  above 

18.  Operation  Orders  (OPORDs)  are: 

_ a.  A  stand-alone  order,  which  facilitates  a  unit's  movement. 

_ b.  The  same  as  an  Operation  Plan  (OPLAN). 

_ c.  Directives  a  Commander  issues  to  subordinate  commanders  to 

coordinate  the  execution  of  an  operation. 

_ d.  Used  in  lieu  of  a  Warning  Order  (WARNO). 

19.  The  two  (2)  general  categories  of  orders  are: 

_ a.  Operations  Orders  and  Warning  Orders. 

_ b.  Garrison  Orders  and  Field  Orders. 

_ c.  Administrative  Orders  and  Combat  Orders. 

_ d.  Commander's  Orders  and  Staff  Orders. 


0 

2 

0 

3 


20.  All  operation  plans  and  orders: 

_ a.  Have  five  (5)  paragraphs;  provide  task  organization  and  scheme 

of  maneuver. 

_ b.  Provide  a  clear,  concise  mission  statement,  based  on  mission 

assigned  from  higher  headquarters. 

_ c.  None  of  the  above 

_ d.  Both  a  and  b 
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21.  All  staff  journals  should  be  archived  using  MARKS. 

_ a.  True 

_ b.  False 

22.  Which  of  the  following  depicts  the  correct  symbol  for  a  demonstration? 


a. 


MASS 


b. 

JOKER 


_c. 

DEMO 


d. 

RIOT 


23.  The  Army  approved  form  for  maintaining  a  journal  within  the  operations  cell  is: 

_ a.  DD1594 

_ b.  DA  1610 

_ c.  DA  1594 

_ d.  DD 1660 


0 

7 

0 

2 


24.  The  data  posted  within  the  journal  should  reflect: 

_ a.  Information  deemed  necessary  for  the  conduct  of  the  operation 

_ b.  Any  orders,  messages,  meetings,  or  meaningful  radio  traffic 

_ c.  Other  criteria  as  outlined  by  unit  SOP  and  the  Commander 

_ d.  All  of  the  above 
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25.  The  governing  protocol  for  the  staff  journal  is: 

_ a.  AR  220-15 

_ b.  Unit  SOP 

_ c.  Conunander's  guidance 

_ d.  All  of  the  above 

26.  The  purpose  of  defensive  operations  is: 

_ a.  Kill  as  many  enemy  as  possible 

_ b.  Resupply  the  soldiers  with  Class  I,  III,  V 

_ c.  Trade  time  and  space  to  gain  sufficient  strength  to  attack 

_ d.  Dig  in  as  deeply  as  possible  and  prepare  for  any  possible  attack 

27.  Retrograde  operations  include: 

_ a.  Delays,  withdrawal,  retirements 

_ b.  Ambush,  attack,  envelopment 

_ c.  Movement  to  contact,  raid,  patrols 

_ d.  Both  b  and  c  are  correct 


0 

5 

1 

0 


28.  Forms  of  maneuver  include: 

_ a.  Envelopment,  turning  movement,  infiltration,  penetration, 

frontal  attack 

_ b.  Raids,  attacks,  deliberate/ hasty,  patrols,  ambush,  pursuit 

_ c.  Both  a  and  b  are  correct 

_ d.  None  of  the  above 
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29.  The  logistic  characteristics  are: 

_ a.  Accountability,  recording,  recordkeeping,  proper  distribution 

_ b.  Anticipation,  integration,  continuity,  responsiveness, 

improvisation 

_ c.  Arming,  repairing,  manning 

_ d.  Anticipation,  integration,  responsiveness,  readiness 

30.  The  term  CEB  refers  to: 

_ a.  Clothing  exchange  bath 

_ b.  Combat  engineer  brigade 

_ c.  Combat  engineer  battalion 

_ d.  Combat  engineer  bridge 

31.  Who  assembles  Logpacs  and  at  what  location? 

_ a.  Unit/TF  Supply  Sgt/ Support  Pit  Ldr/  HHC  Commander  at  the 

field  trains 

_ b.  CSM/TF  Supply  Sgt/First  Sgt/TF  Executive  Officer  in  garrison 

_ c.  Individual  soldiers  supervised  by  the  leader  in  the  combat  trains 

_ d.  Mechanics  under  the  supervision  of  the  BMO  in  the  motorpool 

32.  Which  correctly  depicts  decision  point  #1? 
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33.  When  conducting  a  briefing  of  any  type,  the  room  should  be  cleared  before-hand 
if  the  subject  is  sensitive  in  nature  or  classified. 

_ a.  True 

_ b.  False 

_ c.  True,  but  only  at  the  discretion  of  the  Commander 

34.  The  three  (3)  recommended  techniques  for  war-gaming  are: 

_ a.  Passage  of  line's  method,  operation  by  force,  reconnaissance  by 

force 

_ b.  Belt  technique,  avenue  in  depth,  box  technique 

_ c.  Close  technique,  rear  technique,  deep  technique 

_ d.  None  of  the  above 

35.  The  synchronization  matrix  method  allows: 

_ a.  Tanks,  trucks,  and  personnel  to  arrive  on  time 

_ b.  The  Operations  Officer  to  have  a  better  idea  of  all  battle  assets 

_ c.  The  staff  to  synchronize  the  COA  across  time  and  space  in 

relation  to  the  enemy  COA 

_ d.  All  assets  to  be  on  the  objective  at  the  same  time 

36.  A  key  tool  in  saving  time  in  the  MDMP  is: 

_ a.  Knowing  the  IPB  process. 

_ b.  Keeping  the  S-3,  XO,  and  other  staff  members  informed. 

_ c.  Issuing  the  Commander's  guidance. 

_ d.  Notifying  the  subordinate  Commanders  that  an  order  is  being 

developed. 
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37.  Message  reference  number  refers  to: 


_ a.  How  many  recipients  will  receive  a  copy  of  the  order 

_ b.  The  sequence  in  which  you  will  number  the  pages  within  the 

OPORD 

_ c.  Internal  control  numbers  assigned  by  the  Unit  Signal  Officer, 

lAW  unit  SOP 

_ d.  The  location  of  the  Headquarters  issuing  the  OPORD 

38.  When  possible,  it  is  recommended  that  a  "Letter  Code"  S-Staff,  M-Map,  D- 
Distributed,  etc.  be  used  for  ease  in  getting  information  to  addressees,  and 
creating  an  audit  trail,  placing  this  in  the  action  taken  column. 

_ a.  True 

_ b.  False 


39.  Persons  authorized  to  sign  and  control  the  staff  journal  are: 

_ a.  Support  PSG,  Support  Platoon  Leader,  and  Battalion 

Maintenance  Officer 

_ b.  RTOs,  Battle  Captain,  Operations  Sgts,  and  those  prescribed  by 

the  unit  SOP 

_ c.  Only  the  TF  Commander 

_ d.  Only  the  TF  CSM 

40.  Which  correctly  depicts  contact  point  #3? 


0 

3 

0 

2 


a. 


CP 

#3 


b. 


I 

p 

B 

0 

4 

0 

7 


M 

B 

0 

8 

0 

4 


41.  Key  terrain  is  described  as: 

_ a.  Any  locality  or  area  where  the  seizure  or  retention  of  it  affords  a 

marked  advantage  for  either  combatant 

_ b.  The  best  place  to  focus  the  reconnaissance  effort 

_ c.  A  key  place  for  ground  surveillance  radar 

_ d.  Where  the  Tactical  Operations  Center  (TOC)  is  placed 

42.  If  giving  an  information  briefing,  the  purpose  is  to  convey  information  to  the 
target  audience  in  a  logical,  professional,  manner. 

_ a.  True 

_ b.  Sometimes 

_ c.  False,  does  not  apply  to  this  type  of  briefing 
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Appendix  B  -  Supervisor  Performance  Rating 
Forms 


Dimension  Definitions  Sheet 


Dimension  1  —  Assist  in  the  Military  Decision-Making  Process  (MDMP) 

Understands  and  participates  in  MDMP.  Responds  to  requests  for  information  regarding  the  MDMP. 

Dimension  2  —  Prepare  Combat  Orders  or  Annexes 

Is  knowledgeable  about  the  5-paragraph  order  format  and  the  different  variations  of  orders. 

Dimension  3  —  Prepare  or  Construct  Graphics  or  Overlays 

Uses  graphics  and  overlays  to  convey  operations  orders.  Understands  and  can  produce  military  symbols 
and  graphics. 

Dimension  4  —  Understanding  Intelligence  Preparation  of  the  Battlefield  (IPB) 

Understands  and  participates  in  the  wargaming  process  and  identifies  courses  of  action  based  on  IPB 
process  and  products.  Gathers  and  uses  available  IPB  products  and  resources. 

Dimension  5  — Assist  in  Planning  of  Army  Operations 

Understands  Army  doctrine  and  military  operations  including:  the  nine  principles  of  war,  tenets  of 
combat  operations;  Battlefield  Operating  Systems  (BOS);  participates  in  the  planning  process. 

Dimension  6  —  Assist  in  the  Planning  and  Execution  of  Combat  Support  (CS)  and  Combat  Service 
Support  (CSS) 

Orchestrates  CS  and  CSS  efforts;  imderstands  the  role,  function,  and  structure  of  CS  and  CSS  in  support 
of  Army  operations. 

Dimension  7  —  Manages  Record  Keeping 

Keeps  records  on  staff  journal  log,  DA  Form  1594,  etc.  in  accordance  with  xmit  Standard  Operating 
Procedure  (SOP);  maintains  the  daily  staff  journal. 

Dimension  8  —  Prepare  and  Conduct  Military  Briefings 

Prepares  and  conducts  military  briefings;  delivers  briefings;  understands  different  types  of  briefings  and 
when  to  apply  them. 
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Rating  Instructions 


This  booklet  contains  eight  task  categories  you  will  use  to  make  performance  ratings  as  part  of  this 
USASMA-sponsored  project  imder  the  direction  of  the  Sergeant  Major  of  the  Army.  Each  category 
contains  Rating  Standards  provided  above  the  seven-point  rating  scale.  These  broad  summary  statements 
describe  proficiency  at  different  effectiveness  levels  that  you  can  use  as  comparison  points  to  help  make 
your  ratings  more  objective. 

Making  Your  Ratings 

For  each  category,  read  the  label  and  rating  standards.  Then,  compare  the  NCO’s  current  effectiveness 
with  the  rating  standards  for  that  category. 

For  example,  if  you  feel  that  the  middle  statements  describe  the  person’s  most  typical  effectiveness, 
choose  a  "4.”  If  the  statements  describing  high  effectiveness  on  the  right  of  the  scale  closely  match  his/her 
most  typical  behavior,  choose  a  rating  of  "7."  Likewise,  if  the  statements  on  the  left  of  the  scale  match  the 
NCO’s  most  typical  effectiveness,  choose  a  rating  of "  1 ." 

If  the  NCO  you  are  rating  behaves  as  described  in  the  low  statements  some  of  the  time  but  performs  like 
the  middle  statements  more  of  the  time,  a  rating  of  "3"  would  be  best.  Similarly,  if  both  the  middle  and 
high  level  statements  describe  the  NCO  at  various  times  but  the  high  statements  are  more  descriptive,  the 
fairest  rating  to  give  him/her  is  probably  a  "6." 

Please  use  these  statements  to  help  make  your  ratings  more  objective. 

Once  you  have  selected  a  rating,  make  your  rating  by  blackening  the  appropriate  circle  on  the 
Performance  Rating  Sheet.  Please  make  no  marks  in  this  booklet. 


Finally,  we  very  much  want  you  to  make  these  ratings  on  the  1-7  scale.  However,  if  a  task  category  is  not 

at  all  part  of  the  ratee’s  job,  you  can  mark  the  Not  Part  of  Job  or  “NPP’  circle  for  that  category. 

Important  Points  to  Remember 

1 .  Try  not  to  give  an  NCO  the  same  rating  for  all  eight  categories.  Most  people  will  perform  well  in 
some  categories  and  less  effectively  in  others.  Your  ratings  should  show  the  NCO's  strengths  and 
weaknesses,  as  appropriate. 

2.  Avoid  being  influenced  by  such  things  as  appearance,  family  background,  and  other  personal 
characteristics  that  are  not  directly  related  to  performance. 

3.  Please  rate  independently  (do  not  confer  with  others). 

4.  The  most  important  point  is  to  make  your  ratings  as  accurate  as  possible.  This  is  the  best  way  to 
help  us  on  this  important  project. 
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1.  Assists  in  the  Military  Decision-Making  Process  (MDMP) 

•  Is  not  knowledgeable  about 

•  Is  generally  knowledgeable 

•  Is  very  knowledgeable 

MDMP. 

about  and  understands 

about  and  thoroughly 

MDMP. 

understands  MDMP. 

•  Reluctantly  participates  in 

•  Effectively  responds  to 

•  Proactively  participates  and 

this  process. 

requests  for  information 
regarding  the  MDMP. 

contributes  to  this  process. 

1  2  3  4  5  6  7 


2.  Prepares  Combat  Orders  or  Annexes 

•  Is  not  familiar  with  or 
knowledgeable  about  the  5- 
paragraph  format  or  its 
variations. 

•  Is  generally  knowledgeable 
about  the  5-paragraph 
format. 

•  Is  highly  knowledgeable 
about  the  5-paragraph 
order  format  and  the 
different  variations  of 
orders. 

•  Often  fails  to  recognize  key 
events;  may  take 
inappropriate  actions  when 
he/she  does  recognize  such 
an  event. 

•  Recognizes  key  events  and 
usually  takes  appropriate 
actions. 

•  Always  anticipates  key 
events  and  consistently 
takes  appropriate  actions. 

1  2 

3  4  5 

6  7 

3.  Prepares  or  Constructs  Graphics  or  Overlays 

•  Is  not  able  to  keep  situation 
map  current. 

•  Can  update  situation  map. 

•  Very  effectively  uses 
graphics  and  overlays  to 
convey  operations  orders. 

•  Does  not  have  adequate 
knowledge  of  military 
s)TObols  and  graphics. 

•  Can  interpret  most  military 
symbols  and  graphics. 

•  Thoroughly  imderstands 
and  can  accurately  produce 
military  symbols  and 
graphics. 

1  2 

3  4  5 

6  7 
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4.  Understands  Intelligence  Preparation  of  the  Battiefieid  (iPB) 

•  Does  not  understand  ff  B 
process  or  products. 

•  Understands  some 
elements  of  IPB  products 
and  the  wargaming 
process. 

•  Very  effectively  uses  all 
available  IPB  products  and 
resources. 

•  Does  not  participate  in  or 
understand  wargaming. 

•  Is  able  to  gather  some 
critical  elements  of  IPB 
information. 

•  Thoroughly  understands 
and  proactively  participates 
in  the  wargaming  process, 
and  is  excellent  at 
identifying  courses  of 
action  based  on  IPB 
process  and  products. 

1  2 

3  4  5 

6  7 

5.  Assists  in  Pianning  of  Army  Operations 

•  Does  not  understand  Army 
doctrine  or  the  full  range  of 
military  operations. 

•  Understands  most  aspects 
of  Army  doctrine  in 
support  of  military 
operations. 

•  Thoroughly  imderstands 
Army  doctrine  and  the  full 
range  of  military 
operations  including:  the 
nine  principles  of  war, 
tenets  of  combat 
operations;  and  Battlefield 
Operating  Systems  (BOS). 

•  Is  unable  to  contribute  to 
the  planning  process. 

•  Contributes  to  the  planning 
process  when  asked  or 
prompted. 

•  Actively  and  effectively 
participates  in  the  planning 
process. 

1  2 

3  4  5 

6  7 
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6.  Assists  in  the  Pianning  and  Execution  of  Combat  Support  (CS)  and 

Combat  Service  Support  (CSS) 

•  Does  not  understand  CS 
and  CSS. 

•  Is  typically  able  to  assist  in 
developing  CS  and  CSS 
plans. 

•  Very  effectively 
orchestrates  CS  and  CSS 
efforts. 

•  Is  unable  to  contribute  to 
the  CS  and  CSS  planning 
process. 

•  Has  a  basic  understanding 
of  the  structure  of  CS  and 
CSS. 

•  Thoroughly  xmderstands 
the  role,  function,  and 
structure  of  CS  and  CSS  in 
support  of  Army 
operations. 

1  2 

3  4  5 

6  7 

7.  Manages  Record  Keeping 

•  Does  not  understand  the 
importance  of  the  staff 
journal. 

•  Usually  keeps  accurate  and 
timely  records  on  Staff 
Journal  Log,  DA  Form 

1594,  etc. 

•  Keeps  highly  accurate  and 
timely  records  on  Staff 
Journal  Log,  DA  Form 

1594,  etc. 

•  Does  not  usually  distribute 
reports,  messages,  etc.,  in  a 
timely  maimer. 

•  Distributes  reports, 
messages,  etc.,  as  required. 

•  Always  distributes  reports, 
messages,  etc.,  to  the  right 
people  in  a  timely  manner. 

•  Often  fails  to  recognize  the 
relevance/urgency  of 
reports,  messages,  etc. 

•  Maintains  staff  journal  in 
accordance  with  unit 
Standard  Operating 
Procedure  (SOP). 

•  Proactively  ensures  the 
integrity  and  clarity  of  the 
daily  staff  journal. 

1  2 

3  4  5 

6  7 
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8.  Prepares  and  Conducts  Military  Briefings 

•  Is  unable  to  prepare  or 
conduct  military  briefings. 

•  Is  able  to  assist  in 
preparing  and  conducting 
military  briefings. 

•  Is  excellent  at  preparing 
and  conducting  military 
briefings;  is  poised, 
confident,  and  professional 
when  delivering  briefings. 

•  Does  not  understand  the 
purposes  of  different  types 
of  briefings  and  when  each 
is  appropriate. 

•  Can  adequately  deliver 
most  types  of  military 
briefings. 

•  Thoroughly  understands 
different  types  of  briefings 
and  when  to  apply  them. 

1  2 

3  4  5 

6  7 

Overall  Effectiveness 

Below  Average 

Fully  Adequate 

Exceptional 

1  2  3  4  5  6  7 
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RateeCode:  _ _  Rater  Code: 

Performance  Rating  Sheet 


Ratee  Name: _ _ 

Length  of  time  you’ve  worked  with  the  ratee: _ Months 


Mark  your  responses  by  darkening  the  appropriate  circle.  You  may  use  a  pencil  or  a  pen. 

Note:  NPJ  =  Not  Part  of  the  Job 

1.  Assists  MDMP 

(©) 

2.  Prepares  Combat  Orders/Annexes 

©©© 

3.  Prepares  Graphics/  Overlays 

©® ® ® ©® ® 

4.  Understands  IPB 

©0® @® © © 

5.  Assists  in  Planning  Army  Operations 

©@® @® ® ® 

6.  Assists  in  CS  and  CSS 

©® ® ® ® ® ® 

7.  Manages  Record  Keeping 

©® ® ® ® ® ® 

<®) 

8.  Prepares/Conducts  Briefings 

©® ® ® ® ® ® 

Overall  Effectiveness 

©@® ® ® ® @ 
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Ratee  Code:  _  Rater  Code: 

Experience  Rating  Sheet 


How  much  experience  has  this  person  had,  in  his/her  present  position,  in  each  of  these  battle  staff 
task  areas? 

(Circle  the  appropriate  response) 


No 

Experience 

Some 

Experience 

But  Not  A  Lot 

Considerable 

Experience 

Assists  MDMP 

1 

2 

3 

4 

5 

Prepares  Combat 
Orders/Aimexes 

1 

2 

3 

4 

5 

Prepares  Graphics/  Overlays 

1 

2 

3 

4 

5 

Understands  PB 

1 

2 

3 

4 

5 

Assists  in  Planning  Army 
Operations 

1 

2 

3 

4 

5 

Assists  in  CS  and  CSS 

1 

2 

3 

4 

5 

Manages  Record  Keeping 

1 

2 

3 

4 

5 

Prepares/Conducts 

Briefings 

1 

2 

3 

4 

5 
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Appendix  C  -  Knowledge  Test  Results 


Dependent  Variable;  Overall  Percentage  on  the  Job  Knowledge  Test 

GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.61 

0.11 

89.00 

2.00 

0.62 

0.15 

55.00 

3.00 

0.63 

0.10 

23.00 

Total 

0.61 

0.12 

167.00 

Tests  of  Between-Subjects  Effects 

Dependent  Variable;  Overall  Percentage  on  the  Job  Knowledge  Test 

Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.09 

3.00 

0.03 

1.95 

0.12 

0.04 

Intercept 

2.25 

1.00 

2.25 

153.13 

0.00 

0.48 

EXPTOT 

0.07 

1.00 

0.07 

5.01 

0.03 

0.03 

GROUP 

0.02 

2.00 

0.01 

0.57 

0.57 

0.01 

Error 

2.40 

163.00 

0.01 

Total 

65.54 

167.00 

Corrected 

Total 

2.48 

166.00 

a  R  Squared  = 

.035  (Adjusted  R  Squared  =  .017) 

Notes:  Group  1  =  Fort  Bliss.  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 


C-1 


Descriptive  Statistics 


Dependent  Variable:  Assist  in  the  Military  Decision-Making  Process  (MDMP)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.47 

0.18 

89.00 

2.00 

0.50 

0.22 

55.00 

3.00 

0.50 

0.17 

23.00 

Total 

0.49 

0.20 

167.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable;  Assist  in  the  Military  Decision-Making  Process  (MDMP)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.08 

3.00 

0.03 

0.70 

0.55 

0.01 

Intercept 

3.57 

1.00 

3.57 

93.11 

0.00 

0.36 

EEMDMP 

0.03 

1.00 

0.03 

0.79 

0.38 

0.01 

GROUP 

0.06 

2.00 

0.03 

0.80 

0.45 

0.01 

Error 

6.24 

163.00 

0.04 

Total 

45.61 

167.00 

Corrected 

Total 

6.33 

166.00 

a  R  Squared  = 

.013  (Adjusted  R  Squared  =  -.005) 

Notes;  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Prepare  Combat  Orders  or  Annexes  (POA)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.68 

0.20 

89.00 

2.00 

0.71 

0.20 

55.00 

3.00 

0.78 

0.15 

23.00 

Total 

0.70 

0.19 

167.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Prepare  Combat  Orders  or  Annexes  (POA)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.33 

3.00 

0.11 

3.02 

0.03 

0.05 

Intercept 

10.62 

1.00 

10.62 

293.39 

0.00 

0.64 

EEPOA 

0.14 

1.00 

0.14 

3.94 

0.05 

0.02 

GROUP 

0.18 

2.00 

0.09 

2.48 

0.09 

0.03 

Error 

5.90 

163.00 

0.04 

Total 

89.00 

167.00 

Corrected 

Total 

6.23 

166.00 

a  R  Squared  =  .053  (Adjusted  R  Squared  =  .035) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Prepare  or  Construct  Graphics  or  Overlays  (GO)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.55 

0.21 

89.00 

2.00 

0.55 

0.23 

55.00 

3.00 

0.58 

0.20 

23.00 

Total 

0.55 

0.21 

167.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Prepare  or  Construct  Graphics  or  Overlays  (GO)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.49 

3.00 

0.16 

3.76 

0.01 

0.07 

Intercept 

3.15 

1.00 

3.15 

72.41 

0.00 

0.31 

EEGO 

0.47 

1.00 

0.47 

10.88 

0.00 

0.06 

GROUP 

0.01 

2.00 

0.00 

0.09 

0.91 

0.00 

Error 

7.09 

163.00 

0.04 

Total 

59.00 

167.00 

Corrected 

Total 

7.58 

166.00 

a  R  Squared  = 

.065  (Adjusted  R  Squared  =  .047) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Understanding  Intelligence  Preparation  of  the  Battlefield  (IPB)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.59 

0.20 

89.00 

2.00 

0.66 

0.23 

55.00 

3.00 

0.61 

0.27 

23.00 

Total 

0.62 

0.22 

167.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variable:  Understanding  Intelligence  Preparation  of  the  Battlefield  (IPB)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.64 

3.00 

0.21 

4.61 

0.00 

0.08 

intercept 

4.74 

1.00 

4.74 

101.92 

0.00 

0.39 

EEIPB 

0.49 

1.00 

0.49 

10.56 

0.00 

0.06 

GROUP 

0.21 

2.00 

0.11 

2.30 

0.10 

0.03 

Error 

7.58 

163.00 

0.05 

Total 

71.75 

167.00 

Corrected 

Total 

8.22 

166.00 

a  R  Squared  =  .078  (Adjusted  R  Squared  =  .061) 


Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  ttie  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable;  Assist  in  Planning  of  Army  Operations  (AO)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.59 

0.26 

89.00 

2.00 

0.68 

0.26 

55.00 

3.00 

0.65 

0.21 

23.00 

Total 

0.63 

0.25 

167.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variable:  Assist  in  Planning  of  Army  Operations  (AO)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.33 

3.00 

0.11 

1.72 

0.17 

0.03 

Intercept 

6.30 

1.00 

6.30 

98.29 

0.00 

0.38 

EEAO 

0.06 

1.00 

0.06 

0.96 

0.33 

0.01 

GROUP 

0.28 

2.00 

0.14 

2.17 

0.12 

0.03 

Error 

10.44 

163.00 

0.06 

Total 

77.04 

167.00 

Corrected 

Total 

10.77 

166.00 

a  R  Squared  = 

.031  (Adjusted  R  Squared  =  .013) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variabie:  Assist  in  the  Pianning  and  Execution  of  Combat  Support  (CS)  and  Combat 

Service  Support  (CSS)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.58 

0.21 

89.00 

2.00 

0.51 

0.25 

55.00 

3.00 

0.42 

0.25 

23.00 

Total 

0.54 

0.24 

167.00 

Note:  Groups  1  &  3  have  t-test  value  of  t=3.080,  p=.003 


Tests  of  Between-Subjects  Effects 


Dependent  Variabie:  Assist  in  the  Pianning  and  Execution  of  Combat  Support  (CS)  and  Combat 

Service  Support  (CSS)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.58 

3.00 

0.19 

3.64 

0.01 

0.06 

Intercept 

4.25 

1.00 

4.25 

79.55 

0.00 

0.33 

EECSS 

0.07 

1.00 

0.07 

1.32 

0.25 

0.01 

GROUP 

0.54 

2.00 

0.27 

5.01 

0.01 

0.06 

Error 

8.70 

163.00 

0.05 

Total 

57.25 

167.00 

Corrected 

Total 

9.28 

166.00 

a  R  Squared  =  .063  (Adjusted  R  Squared  =  .046) 


Notes;  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 

Dependent  Variable:  Manages  Recordkeeping  (RR)  Percentage 

GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.84 

0.17 

89.00 

2.00 

0.84 

0.20 

55.00 

3.00 

0.86 

0.15 

23.00 

Total 

0.84 

0.18 

167.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Manages  Recordkeeping  (RR)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.04 

3.00 

0.01 

0.37 

0.77 

0.01 

Intercept 

9.27 

1.00 

9.27 

295.40 

0.00 

0.64 

EERR 

0.03 

1.00 

0.03 

0.87 

0.35 

0.01 

GROUP 

0.00 

2.00 

0.00 

0.07 

0.94 

0.00 

Error 

5.12 

163.00 

0.03 

Total 

124.20 

167.00 

Corrected 

Total 

5.15 

166.00 

a  R  Squared  =  .007  (Adjusted  R  Squared  =  -.011) 


Notes;  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variabie:  Prepare  and  Conduct  Military  Briefings  (MB)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

0.75 

0.24 

89.00 

2.00 

0.68 

0.22 

55.00 

3.00 

0.74 

0.27 

23.00 

Total 

0.72 

0.24 

167.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Prepare  and  Conduct  Military  Briefings  (MB)  Percentage 


Source 

Type  ill  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

0.17 

3.00 

0.06 

1.03 

0.38 

0.02 

Intercept 

6.40 

1.00 

6.40 

113.93 

0.00 

0.41 

EEMB 

0.00 

1.00 

0.00 

0.00 

0.95 

0.00 

GROUP 

0.17 

2.00 

0.08 

1.49 

0.23 

0.02 

Error 

9.16 

163.00 

0.06 

Total 

97.00 

167.00 

Corrected 

Total 

9.33 

166.00 

a  R  Squared  =  .019  (Adjusted  R  Squared  =  .001) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Appendix  D  -  Supervisory  Performance 
Rating  Results 


Descriptive  Statistics 


Dependent  Variable;  Average  of  Supervisory  Performance  Ratings  Across  ali  Eight  Dimensions 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.10 

0.74 

61.00 

2.00 

5.13 

1.04 

38.00 

3.00 

4.74 

0.89 

14.00 

Total 

5.07 

0.87 

113.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable;  Average  of  Supervisory  Performance  Ratings  Across  ali  Eight  Dimensions 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

8.44 

3.00 

2.81 

4.01 

0.01 

0.10 

Intercept 

68.95 

1.00 

68.95 

98.21 

0.00 

0.47 

EXPTOT 

6.76 

1.00 

6.76 

9.63 

0.00 

0.08 

GROUP 

2.21 

2.00 

1.11 

1.58 

0.21 

0.03 

Error 

76.53 

109.00 

0.70 

Total 

2986.18 

113.00 

Corrected 

Total 

84.97 

112.00 

a  R  Squared  =  .099  (Adjusted  R  Squared  =  .075) 


Notes;  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Assist  in  the  Military  Decision-Making  Process  (MDMP)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

4.84 

1.22 

57.00 

2.00 

4.88 

1.01 

34.00 

3.00 

4.50 

1.31 

12.00 

Total 

4.82 

1.16 

103.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Assist  In  the  Military  Decision-Making  Process  (MDMP)  Percentage 


Source 

Type  ill  Sum 
of  Squares 

Df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

10.48 

3.00 

3.49 

2.72 

0.05 

0.08 

Intercept 

118.63 

1.00 

118.63 

92.47 

0.00 

0.48 

EEMDMP 

9.09 

1.00 

9.09 

7.09 

0.01 

0.07 

GROUP 

1.42 

2.00 

0.71 

0.55 

0.58 

0.01 

Error 

127.01 

99.00 

1.28 

Total 

2526.00 

103.00 

Corrected 

Total 

137.50 

102.00 

a  R  Squared  =  .076  (Adjusted  R  Squared  =  .048) 


Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Prepare  Combat  Orders  or  Annexes  (POA)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.20 

0.99 

55.00 

2.00 

4.97 

1.55 

37.00 

3.00 

4.64 

1.69 

11.00 

Total 

5.06 

1.30 

103.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variable:  Prepare  Combat  Orders  or  Annexes  (POA)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

13.96 

3.00 

4.65 

2.92 

0.04 

0.08 

Intercept 

231.31 

1.00 

231.31 

145.22 

0.00 

0.60 

EEPOA 

10.63 

1.00 

10.63 

6.67 

0.01 

0.06 

GROUP 

2.65 

2.00 

1.33 

0.83 

0.44 

0.02 

Error 

157.69 

99.00 

1.59 

Total 

2807.00 

103.00 

Corrected 

Total 

171.65 

102.00 

A  R  Squared  =  .081  (Adjusted  R  Squared  =  .053) 

Notes;  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variabie;  Prepare  or  Construct  Graphics  or  Overlays  (GO)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.51 

1.22 

55.00 

2.00 

5.42 

1.44 

36.00 

3.00 

5.57 

1.28 

14.00 

Total 

5.49 

1.29 

105.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variable:  Prepare  or  Construct  Graphics  or  Overlays  (GO)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

12.04 

3.00 

4.01 

2.50 

0.06 

0.07 

Intercept 

192.62 

1.00 

192.62 

119.95 

0.00 

0.54 

EEGO 

11.74 

1.00 

11.74 

7.31 

0.01 

0.07 

GROUP 

0.04 

2.00 

0.02 

0.01 

0.99 

0.00 

Error 

162.19 

101.00 

1.61 

Total 

3334.00 

105.00 

Corrected 

Total 

174.23 

104.00 

a  R  Squared 

=  .069  (Adjusted  R  Squared  =  .041) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variabie;  Understanding  Inteiiigence  Preparation  of  the  Battiefieid  (IPB)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

4.45 

1.17 

55.00 

2.00 

4.59 

1.46 

32.00 

3.00 

4.23 

1.09 

13.00 

Total 

4.47 

1.25 

100.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variabie:  Understanding  Inteiiigence  Preparation  of  the  Battiefieid  (IPB)  Percentage 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

9.92 

3.00 

3.31 

2.19 

0.09 

0.06 

Intercept 

165.78 

1.00 

165.78 

109.77 

0.00 

0.53 

EEIPB 

8.67 

1.00 

8.67 

5.74 

0.02 

0.06 

GROUP 

1.35 

2.00 

0.67 

0.45 

0.64 

0.01 

Error 

144.99 

96.00 

1.51 

Total 

2153.00 

100.00 

Corrected 

Total 

154.91 

99.00 

a  R  Squared  =  .064  (Adjusted  R  Squared  =  .035) 


Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variabie:  Assist  in  Pianning  of  Army  Operations  (AO)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.08 

1.04 

59.00 

2.00 

4.86 

1.42 

36.00 

3.00 

4.31 

1.18 

13.00 

Total 

4.92 

1.21 

108.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  Assist  in  Planning  of  Army  Operations  (AO)  Percentage 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

22.36 

3.00 

7.45 

5.79 

0.00 

0.14 

Intercept 

150.54 

1.00 

150.54 

116.93 

0.00 

0.53 

EEAO 

15.76 

1.00 

15.76 

12.24 

0.00 

0.11 

GROUP 

6.74 

2.00 

3.37 

2.62 

0.08 

0.05 

Error 

133.89 

104.00 

1.29 

Total 

2767.00 

108.00 

Corrected 

Total 

156.25 

107.00 

a  R  Squared 

=  .143  (Adjusted  R  Squared  =  .118) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable;  Assist  in  the  Planning  and  Execution  of  Combat  Support  (CS)  and  Combat 

Service  Support  (CSS)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

4.87 

1.01 

52.00 

2.00 

5.11 

1.55 

28.00 

3.00 

4.54 

1.45 

13.00 

Total 

4.89 

1.26 

93.00 

Tests  of  Between*Subjects  Effects 


Dependent  Variable:  Assist  in  the  Planning  and  Execution  of  Combat  Support  (CS)  and  Combat 

Service  Support  (CSS)  Percentage 


Source 

Type  III  Sum 
of  Squares 

Df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

10.81 

3.00 

3.60 

2.39 

0.07 

0.08 

Intercept 

185.97 

1.00 

185.97 

123.42 

0.00 

0.58 

EECSS 

7.85 

1.00 

7.85 

5.21 

0.03 

0.06 

GROUP 

3.87 

2.00 

1.93 

1.28 

0.28 

0.03 

Error 

134.11 

89.00 

1.51 

Total 

2371.00 

93.00 

Corrected 

Total 

144.93 

92.00 

a  R  Squared  =  .075  (Adjusted  R  Squared  =  .043) 


Notes:  Group  1  =  Fort  Bliss.  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Descriptive  Statistics 


Dependent  Variable:  Manages  Recordkeeping  (RR)  Percentage 

GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.61 

1.24 

54.00 

2.00 

5.89 

1.16 

38.00 

3.00 

5.00 

1.35 

13.00 

Total 

5.64 

1.24 

105.00 

Note:  Groups  2  &  3  have  a  t-test  value  of  1 .958,  p=.055 


Tests  of  Between-Subjects  Effects 

Dependent  Variable:  Manages  Recordkeeping  (RR)  Percentage 

Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

16.68 

3.00 

5.56 

3.91 

0.01 

0.10 

Intercept 

155.37 

1.00 

155.37 

109.30 

0.00 

0.52 

EERR 

8.84 

1.00 

8.84 

6.22 

0.01 

0.06 

GROUP 

10.47 

2.00 

5.24 

3.68 

0.03 

0.07 

Error 

143.57 

101.00 

1.42 

Total 

3498.00 

105.00 

Corrected 

Total 

160.25 

104.00 

a  R  Squared  =  .104  (Adjusted  R  Squared  =  .077) 


Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate).  - 
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Descriptive  Statistics 


Dependent  Variable;  Prepare  and  Conduct  Military  Briefings  (MB)  Percentage 


GROUP 

Mean 

Std.  Deviation 

N 

1.00 

5.29 

1.23 

59.00 

2.00 

5.28 

1.14 

32.00 

3.00 

5.00 

1.57 

14.00 

Total 

5.25 

1.25 

105.00 

Tests  of  Between-Subjects  Effects 


Dependent  Variable;  Prepare  and  Conduct  Military  Briefings  (MB)  Percentage 


Source 

Type  III  Sum 
of  Squares 

df 

Mean 

Square 

F 

Sig. 

Eta  Squared 

Corrected 

Model 

10.93 

3.00 

3.64 

2.44 

0.07 

0.07 

Intercept 

117.33 

1.00 

117.33 

78.67 

0.00 

0.44 

EEMB 

9.94 

1.00 

9.94 

6.67 

0.01 

0.06 

GROUP 

0.93 

2.00 

0.46 

0.31 

0.73 

0.01 

Error 

150.63 

101.00 

1.49 

Total 

3053.00 

105.00 

Corrected 

Total 

161.56 

104.00 

a  R  Squared  =  .068  (Adjusted  R  Squared  =  .040) 

Notes:  Group  1  =  Fort  Bliss,  Group  2  =  VTT,  Group  3  =  Fort  McCoy  (used  as  the  Independent  Variable). 
EXPTOT  =  Average  experience  rating  from  the  NCO  (used  as  a  covariate). 

EE  prefix  in  front  of  a  dimension  signifies  NCO  experience  ratings  (used  as  a  covariate). 
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Appendix  E  -  Supervisors’  Comments 


Comments  from  Onsite  Visits 

This  coxirse  was  very  helpful  to  this  soldier.  He  had  purely  Artillery-based  background  before 
this.  This  course  helped  him  to  xmderstand  what  needed  to  happen  in  an  S-2  Shop.  This  course 
helped  our  shop  during  our  Warfighter  exercise. 

This  soldier  has  been  a  96B  for  quite  some  time.  However,  the  course  seemed  to  be  very 
beneficial  to  him  because  of  the  areas  on  planning  and  briefing  that  were  covered. 

Should  be  taught  at  the  ANCOC  level. 

Should  be  taught  at  the  ANCOC  level. 

This  division  was  very  successful  in  its  Warfighter  because  of  these  outstanding  Staff  NCOs. 

I  cannot  stress  the  importance  of  these  NCOs  enough.  They  run  tiie  TCOs/  ALOCs/ CPs  for  the 
Army.  This  school  is  invaluable  to  the  C2  process  of  the  Army.  As  both  an  Infantry  and 
Intelligence  Officer,  I  spent  most  of  my  career  in  TOCs  either  working  or  observing/  controlling 
at  JRTC.  One  can  immediately  teU  the  difference  between  BSNCO  qualified  and  non-BSNCO 
qualified  NCOs.  They  are  absolutely  critical  to  the  success  of  the  Army  especially  as  we  move 
into  the  21st  century.  I  strongly  recommend  that  aU  NCOs  in  the  rank  of  SSG  to  MSG  attend 
this  course  and  make  it  a  prerequisite  to  promotion  to  E-9.  This  course  is  the  CAS3  for  NCOs 
and  is  extremely  important. 

Many  slots  are  directed  by  DA.  Shorter  versions  of  the  class  shoxdd  be  given  so  more  NCOs  can 
attend!  Junior  NCOs  should  be  given  this  training  to  better  them  for  leadership  roles.  I've 
heard  notiung  but  great  things  from  this  course. 

The  knowledge  that  is  passed  on  by  my  subordinate  is  very  well.  Whatever  you're  teaching  is 
good. 

There  is  a  considerable  difference  in  experience,  knowledge,  and  confidence  between  those  who 
graduated  from  the  "short"  course  and  the  seven-week  Bliss  course.  Those  grads  from  Bliss  had 
a  better  working  knowledge  of  Army  Ops  and  Graphics.  The  short  cotuse  doesn't  seem  to 
provide  the  "hands-on"  of  the  long  course.  The  short  course  depends  considerable  on  the  home 
study  section  and  individuals  vary  on  how  well  they  handle  and  absorb  knowledge  using  ttiis 
method.  NCOs  have  been  indoctrinated  and  feel  comfortable  with  resident  courses  and  this 
could  be  a  factor. 

In  12  months  of  battalion  operations  I  have  been  a  supervisor  for  4  separate  NCOs  who  have 
graduated  from  the  Battle  Staff  Course.  Without  exception,  I  have  seen  a  glaring  deficiency  in 
the  ability  of  Battle  Staff  NCOs  to  manage  a  battalion  TOC  efficiently.  Understand  that  aU  four 
NCOs  are  solid  performers,  but  developing  battalion  OPORDs  in  a  constrained  environment  is 
not  understood.  Many  leaders  expect  NCOs  to  come  back  from  this  course  with  answers  on 
how  to  operate  a  TOC  in  the  field.  This  is  the  number  one  expectation,  and  appears  to  be  the 
least  of  topics  addressed. 


Comments  from  the  Mail  Version 

As  a  graduate  of  Ae  Battle  Staff  course  MSG  *****,  exudes  confidence  and  is  always  well 
prepared  to  brief  the  Battalion  Commanders  training  guidance  and  disseminate  all  the  tactical 
information  to  the  Batteries,  MSG  *****  is  an  outstanding  BSNCO  who  demonstrates  the  ability 
to  manage  a  number  of  critical  tasks  simultaneously.  MSG  *****  would  be  an  excellent 
instructor  for  Battle  Staff,  his  professional,  intelligent  epitomizes  what  every  BSNCO  should  be. 


The  training  that  NCO's  receive  at  the  Battle  Staff  NCO  course  is  outstanding.  It  strengthens 
the  team  that  is  planning,  preparing,  and  briefing  a  myriad  of  operations.  Good  staff  NCOs 
going  in  are  great  NCO's  contributing  to  the  staff  process  positively  coming  out.  Please 
continue  to  screen  and  training  quality  staff  NCO s. 


It  has  been  my  experience  that  senior  NCO  utilizes  the  skiUs  they  have  learned  at  BSNCO 
Course.  In  particular  the  previous  job  experience  i.e.  PSG's  senior  squad  leaders  and  company 
master  gunners  is  critical  towards  becoming  an  integral  part  of  any  staff.  They  have  a 
knowledge  of  "I  know  what  we  need!"  When  an  NCO  has  no  experience  at  the  line  company 
level  how  will  they  know  what  works. . .It  is  hke  a  group  of  OFFICERS  formulating  a  packing 
list  for  deployment  despite  they  fact  tiiey  have  never  "humped"  w/a  ruck!! 

THE  NEXT  LEVEL  of  education  should  mirror  ffie  military  science  taught  the  SENIOR 
OFFICER  CORP.  The  dedication  and  knowledge  our  senior  NCO's  possess,  why  wotdd  we  not 
TAP  that  experience  and  teach  the  science/art  of  war?  We  can  handle  a  bit  more  than  "Head 
Count"!! 


Depending  on  each  units  manning  situation,  and  battle  staff  structure,  some  BSNCO's  may  not 
have  the  full  opportunity  to  exercise  what  they' ve  learned.  My  comments  were  based  both  on 
performance  and  potential.  In  many  cases,  squadron/ Battalion  MDMP  is  conducted  only  at  the 
primary  staff  level,  with  BSNCO's  providing  support  for  current  operations. 

Provide  students  with  a  smartbooks  for  tiiose  NCOs  who  will  occupy  S-S  OPS  and  S2  NCOIC 
slotted  positions.  Provide  students  with  a  CD-ROM  or  2S  floppy  disk  of  current  military 
symbols  used  in  BN  Operations  and  higher. 

SFC  *****  has  a  better  understanding  of  the  orders  process  based  on  his  Battle  Staff  schooling. 
This  is  vital  to  an  O/  C. 

SSG  *****  has  put  some  of  this  to  use  but  he  is  an  Observer  Control  Trainer  for  National  Guards 
units  and  rarely  get  the  time  to  use  what  he  has  learned  in  Battle  Staff. 

I  went  through  one  LTP  at  JRTC  with  this  NCO.  Out  of  six  NCOs  in  my  shop  he  has  the  highest 
level  of  xmderstanding  on  what  an  OPORD  is  and  how  to  help  write  one.  SFC  *****  does  not 
understand  the  content  and  information  that  needs  to  be  written  into  the  order.  We  discussed 
the  Battle  Staff  Course  and  he  informed  me  that  his  training  mainly  focused  on  .the  logistics 
portion  of  planning  and  operations.  I  had  higher  expectation  concerning  his  ability  to  assist  in 
the  entire  orders  writing  process. 
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Comments  from  the  Mail  Version  (continued) 


Items  4  (IPB),  5  (Planning  Army  Operations)  &  8  (Prepares/ Conducts  Briefings)  of  the 
Performance  Rating  Sheet  (Enel  4)  could  not  be  accurately  assessed  for  this  NCO.  Though  I  am 
extremely  confident  in  the  BSNCO  training,  I  have  not  had  the  opportunity  to  witness  this 
NCO's  ability  to  effectively  participate  in  items  4, 5  &  8. 

The  reason  for  this  is  due  to  the  lack  of  manpower  in  our  BN  S4  section.  With  few  exceptions, 
our  section  has  been  manned  solely  by  the  S4  NCOIC  and  conducted  by  the  S4  OIC  while  the  S4 
NCOIC  was  executing  day  to  day  operations  keeping  this  section  operational. 


I  think  the  course  is  worthwhile  provided  the  Chain  of  Command  permits  the  NCO  full 
opportunity  to  concentrate  on  the  course  requirements.  Several  units  do  not  allow  the  NCOs 
fiiU  concentration  on  the  course  and  expect  ^e  NCO  to  perform  daily  duties;  not  fair  to  the 


NCO. 


No  comments.  Course  was  well  balanced  in  all  areas.  *****  wins  professional.  *****  support 
exceptional. 

I  think  the  BSNCO  course  fully  trained  SSG  *****  for  all  the  tasks  in  this  survey.  When  he 
arrived  from  the  course,  he  was  motivated  and  eager  to  contribute  to  tiie  S3  section.  However, 
officers  do  not  feel  an  NCO  know  enough  about  these  tasks  and  hold  on  to  that  life  their  lives 
and  depend  on  it.  As  long  as  an  NCO  can  prepare  graphics,  get  up  and  tear  down  a  TOC  and 
make  sure  they're  fed,  they  feel  that  is  a  BSNCO  that  is  fully  capable.  1  don't  agree  and  I've 
tried  to  implement  changes,  but  that  seems  to  be  the  status  quo  in  this  unit.  We  do  have  new 
CDR  and  S3  that  is  willing  to  include  NCOs  completely  in  aU  aspects  of  planning  on  the  staff 
and  I'm  encouraged  by  that.  But  up  until  now,  *****  has  had  little  opportunity  to  prove 
himself  to  be  capable  and  fulfill  his  potential. 


This  NCO  is  now  working  in  a  Joint  Command.  While  much  of  the  course  is  applicable  to  the 
small  units  at  the  tactical  level  very  little  transfers  to  the  "Joint"  strat/ operational  levels.  This 
cotirse  wiU  serve  him  well  when  he  returns  to  the  Regular  Army. 


Note:  Names  and  inappropriate  comments  were  deleted. 
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Appendix  F  -  Graduates’  Comments 


Comments  from  Onsite  visits 

Battle  Staff  conducted  at  Ft.  Bliss  diiring  a  6-week  period  was  effective.  3  weeks  saves  $, 
however,  students  need  time  to  absorb  &  retain  as  opposed  to  the  "brain  dump"  during  a  3- 
week  crash  course.  I  strongly  oppose  the  VTT  course  of  instruction  due  to  students  not  having 
actual  "hve"  face  to  face  with  instructors  at  Ft.  Bliss. 

The  course  helped  me  to  understand  how  higher  headquarters  work  in  relation  to  other  units  & 
branches  of  title  military.  I  feel  that  one  should  really  be  used  more  as  a  Battle  Staff  NCO  to 
maintain  the  knowledge  received  from  the  course. 

Battle  Staff  at  Ft.  McCoy  is  too  short.  It  did  not  allow  for  real  learning.  Only  how  to  reference 
material  was  any  help.  The  course  was  canned.  Giving  out  already  produced  orders  instead  of 
developing  our  own.  I  imderstand  tihe  Active  Component  does  not  use  this  course  any  longer 
. . .  That's  Good!  A  Battle  Staff  should  conclude  with  a  real  field  exercise.  Cover  installation  of  a 
TOC  (Tactical  Operation  Center).  TOC  Battle  Drills,  Receive  higher  HQ  order,  go  through  the 
MDlvff  process  completely.  Jump  TOC  imder  fire  and  not  imder  fire  conditions.  Force 
students  to  use  the  file  journal.  Make  them  look  up  old  information  in  the  file  journal  to  see  if 
they  can.  Make  them  go  through  shift  change  briefings,  and  finally  recovery  tiie  TOC  at  the 
conclusion.  Even  try  to  recover  while  doing  their  jobs  in  the  S-2/  ^3  Shop  planning  for  the  next 
6-8  weeks  out.  Most  units  put  so  much  on  our  plate  that  recovery  suffers  and  military 
equipment  breaks  down  and  when  we  really  need  it ...  It  won't  be  there  or  it  won't  work! 

Battle  Staff  is  not  useful  to  the  military  and  will  not  be  useful  to  the  military  until  the  NCOs  are 
given  a  larger  role  within  the  Battle  Staff  operation.  Too  often  I  see  NCOs  charged  with  the 
menial  tasks  around  the  operation  while  the  officers  wargame  &  MDMP.  The  NCOs  have  years 
of  experience  on  the  battlefield  and  should  be  used  for  that  knowledge.  The  school  taught  me  a 
great  deal  about  graphics,  but  I  don't  feel  that  there  was  enough  info  on  maneuvering  units 
tactically. 

My  personal  experience  with  the  course  was  very  good.  I  want  to  applaud  the  decision  on 
insuring  that  all  25th  ID  (LT)  soldiers  attend  the  Pre-Battle  Staff  Course.  It  definitely  assisted 
me  on  the  graphics  however  the  course  should  have  had  a  more  in  depth  overview  over  the 
tested  topics.  In  regards  to  the  Distance  Learning,  I  personally  don't  feel  that  it  would  be  a  good 
idea  for  this  course  if  it  is  going  to  be  expected  to  be  retained.  This  course  is  definitely  designed 
to  be  used  on  a  continual  basis  back  at  the  unit  level.  Due  to  the  fact  that  I  was  at  the  company 
level,  I  really  didn't  have  an  opportunity  to  exercise  some  other  avenues  that  the  course  taught 
me.  Overall  it  has  been  the  best  military  school  I  have  attended  in  regards  to  knowledge. 

Course  can  be  shortened  and  thus  save  $  by  making  Saturday  a  work  day.  BSNCOC  should  not 
be  changed.  Small  group  sessions  are  great  and  by  going  TDY  the  distracters  of  hone  units  are 
gone.  The  course  relies  too  much  on  "open"  book  tests.  In  real  life,  we  do  not  have  manuals  on 
hand  or  time  to  reference  them. 

I  feel  that  NCOs  should  go  earlier  in  their  career  (6  yr)  or  need  to  be  included  in  BNCOC. 


Comments  from  Onsite  visits  (continued) 

Great  school,  however  NCOs  are  not  used  like  they  should  upon  return  to  their  unit. 


Course  FTX  exercise  must  provide  realistic  scenarios.  Must  be  conducted  outdoors  not  indoors. 

My  attendance  to  the  Battle  Staff  Course  was  one  of  the  most  rewarding  schools  I  have 
attended.  I  learned  a  lot  about  what  goes  into  planning  an  operation  and  support  required  for 
sustainment.  As  a  result,  I  am  better  qualified  to  assist  in  the  planning  process  at  a  level  that  I 
couldn't  before.  I  think  that  the  facilities  at  Ft.  Bliss  and  the  class  environment  is  very  conducive 
to  the  Battle  Staff  program. 


Battle  Staff  helped  me  become  an  integral  part  of  the  Staff  during  the  Divisional  Warfighter.  It 
helped  me  to  become  aware  of  the  entire  depth  of  the  battlefield  enabling  me  to  become  an 
invaluable  asset  during  the  MDMP  process  and  inside  the  TOC.  As  our  officers  are  planning 
future  operations  it  is  mandatory  that  Battle  Staff  qualified  NCOs  constantly  track  and  manage 
the  entire  depth  of  the  battlefield  or  battle. 

The  BSNCOC  was  one  of  the  most  informative  courses  I've  completed.  The  course  made  it 
possible  for  the  senior  NCOs  to  be  proficient  in  tracking  the  battle.  The  course  allows  the  NCO 
to  track  the  battle  without  the  help  of  the  officer  and  allows  the  officer  to  make  wiser  decisions 
about  the  battle  based  on  the  input  from  the  NCO.  The  course  is  very  beneficial  to  soldiers  in 
MOSs  other  than  combat  arms.  I've  been  on  both  sides  of  the  field.  Combat  arms  soldiers  get  a 
lot  of  this  type  of  experience  from  their  daily  jobs.  Support  soldiers  are  rarely  familiar  with 
battle  staff  operations  unless  they  are  placed  in  a  battle  staff  position  or  attend  the  battle  staff 
course.  When  I  attended  the  course,  a  lot  of  actions  I  had  previously  experienced  due  to  being 
an  IIB  in  my  earlier  years  of  the  mBitaiy.  Now  that  I'm  a  CSS  soldier  I  find  that  a  lot  of  my 
peers  have  no  clue.  Would  have  loved  to  work  in  a  battle  staff  position  for  longer  and  put  my 
experience  to  use. 

First,  this  test  seemed  to  lean  heavily  towards  administrative  warfighting.  Most  of  the  topics 
were  not  covered  in  the  course.  Most  importantly,  let's  train  junior  NCOs.  A  senior  NCO  is 
already  in  position  and  expected  to  perform  from  day  one.  Let's  focus  on  the  MOS  of  CMF  that 
is  most  likely  to  be  asked  to  write  plans,  orders  or  annexes.  For  example,  a  SSG  (13F30)  is  neck 
deep  in  the  orders  process  at  the  company  level.  Let's  Battle  Staff  train  him. 

Emphasis  on  the  actual  role  in  the  MDMP.  Less  emphasis  on  Dimension  2, 7  &  8.  [He  is 
rejerring  to  a  Definition  Dimension  Sheet  we  passed  out  to  remind  them  of  the  course  content. 
Dimensions  2,  7  and  8  are:  Prepares  Comhat  Orders/Annexes,  Manages  Record  Keeping,  and 
Prepares/Conducts  Briefings.]  Emphasize  what  only  a  trained  Battle  Staff  NCO  can  do. 
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Comments  from  Onsite  visits  (colitinued) 

I  taught  the  Hawaii  Pre-Battle  Staff  Course  which  was  designed  to  help  soldiers  tmderstand 
what  they  needed  to  accomplish  at  the  resident  course.  I  found  that  when  looking  at  the  total 
lesson  plans  of  the  actual  Battle  Staff  Course  that  not  enough  instruction  was  dedicated  to  filling 
out  journals  or  battle  logs.  While  lots  of  time  was  dedicated  to  what  would  be  primarily  officer 
based  jobs  at  Division  Staff  Level,  minimum  time  was  given  to  how  to  set  up  shifts,  personnel 
management,  TOC  set  up;  things  that  generally  fall  on  the  senior  enlisted.  Individual  NCOs 
who  never  worked  in  a  TOC  fail  these  tasks  because  of  lack  of  this  knowledge.  An  additional 
overview  of  these  tasks  may  help  in  tiie  future. 


Ft.  McCoy:  Three  straight  weeks  was  fast  and  furious.  Great  course  to  better  enhance  the  skills 
needed  to  fight  and  win  the  battle  at  the  staff  level  and  also  good  for  future  use  in  an  infantry 
platoon  to  pass  on  to  the  yoxmger  soldier  in  order  to  aid  him  in  symbols  used  on  a  map. 
Definitely  need  to  send  soldiers  to  a  pre-coxirse  at  home  station  before  going  to  the  3-week 
course. 


Most  of  the  reference  materials  were  too  outdated  and  not  related  to  present  mode  of  operation. 
Example:  Soviet  doctrine. 


All  cotirses  should  have  weekends  off  -  so  students  can  clear  their  heads.  But  it  was  still  a  good 
cotuse. 


The  Battle  Staff  NCO  is  a  valuable  asset  to  the  Battle  Staff.  With  the  strengths  of  knowledgeable 
NCOs  in  TOCs/TACs/CPs  the  overall  capabilities  of  a  commander  to  command  and  control  his 
unit  is  expotmded  greatly.  You  can  absolutely  see  the  difference  between  a  staff  that  has  well 
trained  BSNCOs  and  one  that  does  not.  This  course  is  critical  to  the  overall  success  of  all 
operations  coordinated  at  any  higher  headquarters.  The  NCOs  who  have  been  trained  through 
the  Battle  Staff  Course  are  better  able  to  compliment  their  officer  coxmterparts  in  the  entire 
realm  of  combat  operations,  and  in  some  cases  exceed  the  capabilities  of  junior  officers.  The 
Army  must  keep  this  course  and  continue  to  develop  it,  as  it  already  has  and  will  continue  to 
better  prepare  NCOs  for  their  roles  in  future  complex/joint  ands  strategic  operations. 


A  pretty  interesting  course.  All  NCOs  and  officers  should  be  provided  the  opportunity  to 
participate  in  the  coxirse.  Hopefully,  with  the  new  distance  learning  program,  it  will  take  away 
some  of  the  stress  and  fear  of  the  course.  Should  be  a  refresher  given  so  Battle  Staff  NCOs  can 
maintain  and  keep  up  with  changing  technology. 


The  bmeting  at  Fort  McCoy,  WI  is  in  bad  need  of  repair  and  is  not  up  to  standards  for  soldiers 
living  there.  No  refrigerator  in  room,  door  without  locks,  and  no  wall  lockers. 

Good  course  overall  except  some  material  was  outdated;  i.e.  manuals.  Some  instructors  wanted 
to  go  home  early.  More  emphasis  needs  to  be  placed  on  MDMP  at  the  course. 
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Comments  from  the  Mail  Version 


I  attended  the  DL  BSNCO  course  at  Fort  Polk  during  die  period  of  Jun-Sep  99.  After  attend  the 
course  I  was  selected  as  an  AI  for  the  CL  02-00.  As  an  AI  and  a  student  I  felt  that  the  DL  course 
is  an  excellent  way  for  the  Army  to  save  on  spending;  however  I  still  believe  that  one  on  one 
training  is  more  beneficial.  One  on  one  training,  in  my  opinion,  enables  me  to  retain  the 
information  a  lot  longer. 

Some  of  the  course  material  i.e.  PE's  was  not  available  or  had  changed  during  both  dass  01-00 
and  CL  02-00. 

After  I  completed  the  course  I  foxmd  it  to  be  of  value  in  performing  my  job  as  an  S3  Operations 
NCO. 

The  instructors  at  the  USASMA  were  quality  subject  matter  experts  who  presented  the  blocks  of 
instruction  so  that  it  could  be  clearly  understood. 


I  expected  more  from  Ft.  Bliss  Instructors.  Most  times  they  didn't'  have  a  clue.  Why  not  fill 
these  positions  with  Subject  Matter  Experts  from  the  area  that  is  being  taught. 

The  course  made  its  best  attempt  at  ramming  20  lbs.  of  *****  in  a  5  lb.  bag.  I  learned  very  little. 

I  usually  feel  a  great  sense  of  accomplishment  when  I  complete  a  course/  school.  I  did  not  feel 
this  way  after  completing  this  course. 

The  course  was  a  waste-except  graphics  &  plans  &  orders. 

Suggestion:  1)  Get  qualified  instructors,  2)  Reduce  the  amoxmt  of  material  (Germany  DL  grad). 

VTT  can  be  more  difficult  to  grasp  aU  the  information  being  presented  based  on  the  instructor 
to  student  ration.  The  majority  of  the  students  I  attended  dass  with  would  have  preferred  to 
attend  six  straight  weeks  at  Ft.  Bliss. 

AU  other  comments  from  my  class  were  provided  directly  to  SGM  *****(chief  instructor  for 
Battle  Stafi)  on  6  April  00  (VTT  graduate). 


The  evaluation  is  a  good  idea,  but  I  feel  that  it  should  have  been  sent  out  sooner,  a  lot  of  the 
areas  of  the  course  are  not  used  in  my  job  so  it  was  difficult  answer  some  of  the  questions,  but  I 
tried  to  answer  tiiem  to  tiie  best  of  my  ability. 

I  also  strongly  feel  that  the  residential  course  is  great  because  it  gets  the  NCO's  away  from  there 
home  station  and  they  can  concentrate  on  the  course  without  any  outside  distractions. 
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Comments  from  the  Mail  Version  (continued) 

The  Battle  Staff  class  I  attended  last  summer  was  good.  The  administration,  billeting, 
evaluation  and  practical  exercise  were  all  good.  I  did  not  have  the  time  to  do  the  pre-resident 
phase.  It  turned  out  to  be  unnecessary  anyway.  I  did  very  weU  without  it. 

The  teleconference  was  a  great  idea.  I'm  sure  it  saved  a  lot  of  money.  It  could  have  saved  even 
more  if  we  had  held  the  class  here  at  Ft.  Campbell  instead  of  sending  us  to  Ft.  Benning. 


The  BSNCO  course  is  a  wonderful  course.  Being  from  a  non-combat  arms  MOS I  learned  a  lot 
from  the  course.  I  was  upset  though  when  they  reduced  the  length  of  the  course.  I  believe  that 
when  I  was  there  for  the  6  weeks  that  I  learned  more  than  I  would  have  if  I  had  to  do 
correspondence  courses  prior  to  attending  the  resident  portion.  There  is  nothing  better  than 
being  able  to  have  group  discussion  with  soldiers  with  aU  different  types  of  backgroxmds  & 
experience.  Being  able  to  learn  from  the  instructors  &  classmates  made  me  a  better  BSNCO.  It 
allowed  me  to  see  other  MOS's  point  of  view  on  situations.  I  appreciate  being  chose  to  take  this 
test. 


I  attended  the  course  at  Ft.  McCoy,  I  foimd  the  pre-resident  material  very  good  but  I  received  it 
only  about  20  days  prior  to  the  course.  More  time  to  complete  this  phase  would  have  been 
helpful. 

The  resident  phase  at  Ft.  McCoy  was  outstanding.  The  content,  materials  &  practical  exercises 
were  relevant  &  to  the  point.  TTie  instructors  were  excellent,  professional  &  extremely  well 
versed  in  their  sul^ect. 


The  pre  resident  (Phase  I)  is  a  must.  It  gives  a  good  overview  of  the  resident  course.  If  students 
complete  phase  I  and  pass  the  test  it  will  greatly  increase  their  chances  for  success.  It  helped 
me.  The  Battles  Staff  NCO  course  is  the  best  and  most  rewarding  school  I  have  completed  in 
my  career.  The  biggest  challenge  is  using  the  Battle  Staff  NCCy  s  in  the  appropriate 
assignments.  I  am  a  13R402S  and  have  not  been  working  in  a  battle  staff  NCO  position  since  I 
graduated  in  July  99.  I  should  be  moving  on  to  a  targeting  position  in  a  Brigade  or  Division 
FSE.  But  there  are  no  authorized  positions  for  a  13R40.  C5ur  only  progression  is  in  on  Target 
Processing  Section  that  is  attached  to  the  Coxmterfine  Headquarters.  Our  training,  in  my 
opmion,  is  being  wasted.  Thank  you  for  the  opportunity  to  give  my  opinions. 


Currently  assigned  as  a  truckmaster  in  a  heavy  equipment  transport  company. 


I  took  the  test  without  any  preparation.  I  used  no  books  or  notes  to  help  me.  I  would  like  to 
know  the  results  if  at  aU  possible. 

All  of  the  subjects  taught  are  valuable  to  leaders.  Coming  out  of  the  course  I  felt  good  knowing 
that  what  I  learned  is  used  in  today^s  day  to  day  business.  I  would  consider  going  to  more 
schools  hke  Battle  Staff.  They  should  have  more  like  this  one  for  our  NCOs. 
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Comments  from  the  Mail  Version  (continued) 

I  think  one  of  the  biggest  problems  my  classmates  and  I  had  were  on  the  intelligence  materials. 
In  many  cases  during  the  class  we  found  that  the  material  were  out  dated  or  wrong.  I  do 
believe  that  areas  needs  a  lot  of  work,  and  if  we  had  received  more  PE^s  on  that  area  we  would 
have  felt  better.  I  passed  that  portion  on  the  first  try,  but  I  felt  that  I  did  not  know  any  more 
than  I  already  knew  and  for  the  most  part  my  classmates  and  I  were  just  lost  (VTT  graduate). 


More  1-imp  shotdd  have  been  allocated  to  the  practical  exercise  "battle"  at  the  end  of  the  course. 
Three  days  is  just  not  enough  time  to  replicate  the  desired  environment.  In  addition  twenty- 
four  hour  operations  would  have  added  realism 

I  felt  that  tire  time  devoted  to  the  MARKS  system  was  a  waste.  That  could  have  been 
adequately  covered  in  the  pre-resident  phase  (VTT  graduate). 

Intelligence  portion  of  the  Battle  Staff  NCO  Course  was  the  most  hardest  portion  of  the  course. 

I  recommend  that  you  can  provide  subject  matter  expert  instructors  on  the  Intel  portion  of  the 
course.  Most  student  on  our  class  was  confused  the  instructor  let  the  student  read  the  book 
word  for  word  just  like  reading  a  newspaper  with  out  explanation  or  sample  situation  on  how 
to  apply  the  four  step  of  IPB. 

It  did  not  seem  to  matter  about  GPA.  As  long  as  one  got  over  70%  tihe  instructors  were  happy. 

It  also  seemed  that  there  were  personal  preferences  in  choosing  who  exceeded  course  standards. 
GPA's  should  be  stated  on  the  1059  during  field  exercise,  some  of  the  positions  had  minor 
involvement  while  others  were  swamped.  The  instructors  need  to  prompt  soldiers  in  doing 
their  tasks. 

I  enjoyed  the  VTC.  It  was  informative.  It  also  seemed  that  at  times  the  instructors  "shot  from 
the  hip"  before  researching. 

Overall  it  was  a  good  course. 


The  course  was  adequate,  insofar  as  it  thoroughly  explained  the  battle  staff  on  a  theoretical 
level.  I  was  disappointed  that  the  course  did  not  address  certain  key  aspects  of  TOC  operations 
essential  to  the  competent  battle  staff  NCO.  The  comrse  focused  on  neither  the  physical 
composition  nor  the  dynamics  that  make  up  the  TOC,  and  are  of  primary  importance  to  the 
battie  staff  NCO. 


Most  of  the  instructors  at  the  BSNCO  course  were  very  knowledgeable  and  cared.  The 
instructor  I  had  was  not.  He  failed  to  satisfactorily  answer  questions  and  we  would  not  rise  him 
for  reviews.  We  did  that  on  our  own. 
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Comments  from  the  MairVereion  (continued)  ^ 

It  seems  that  the  NCO's  who  attended  the  BNSCO  were  primarily  concerned  with  their  area  of 
expertise  so  they  could  better  their  skills  when  they  got  back  to  their  unitSi  Most  of  the  subject 
matter  that  didn't  closely  relate  to  their  jobs  was  flushed  from  their  brains  after  the 
exams/course. 

There  was  a  double  standard  for  the  test  taking.  There  was  one  NCO  who  was  allowed  to 
retake  a  test  several  times.  The  standard  was  one  re-take.  I  felt  that  when  that  NCO  challenged 
the  test  the  NCO  shouldn't  have  been  able  to  retake  it.  That  NCO  had  the  same  opportunity  as 
every  one  else  to  pass  the  test  in  the  retake.  I  know  that  I  would  question  that  NCCy  s  abilities 
when  returned  to  the  parent  unit.  Am  I  too  hard?  Do  1  not  xmderstand  the  whole  situation?  I 
think  appearance  says  a  lot  and  there  were  a  lot  of  rumors  floating  around  about  the  situation. 


A  lot  of  NCO's  I  attended  with  were  not  in  Battle  Staff  Positions.  I  had  to  wait  a  long  time  to  get 
the  slot  because  of  this.  Paperwork  was  changed  (i.e.  ERB  2-1)  to  put  them  in  a  2S  slot 
temporarily,  just  to  go  to  school.  This  has  been,  and  still  is  being  abused  at  all  levels. 

Other  than  fliat. .  .the  course  was  great,  I  thought  my  instructor  was  very  knowledgeable  and 
professional. 


I  enjoyed  the  coxuse  and  feel  that  it  is  very  beneficial  to  all  NCO's.  It  taught  me  the  total  Army 
concept  by  allowing  me  to  interact  w/ other  senior  NCO' s  about  how  to  prepare  for  and 
conduct  combat  operations.  I  have  found  out  that  it  is  a  perishable  skill  that  needs  to  be  worked 
constantly. 


It  was  a  very  good  course.  It  helped  me  very  much  because  I  was  given  the  job  as  the  Battalion 
Intel.  Sergeant  of  a  newly  formed  Battalion  and  I  had  to  start,  from  the  groxmd  floor  to  build  the 
S2  section  into  an  integral  part  of  tiie  BN  staff.  I  xised  much  of  what  I  learned  many  times 
during  field  training  and  OPORD  prep. 


My  comments  refer  to  the  picking  of  the  top  20%  of  the  class.  I  think  it  should  be  relooked  how 
this  is  done.  I  was  not  picked  the  top  20%  of  my  small  group  of  instruction,  but  1  would  have 
been  in  almost  all  other  groups.  My  point  is  it  should  be  conducted  picking  the  entire  class  not 
just  from  your  group.  Example:  Academic  average  of  98%  doesn't  get  you  in  top  20%  in  my 
group,  but  in  another  group  93%  got  you  top  20%. 

Second  topic  -  the  course  does  not  teach  practical  battle  staff  stuff.  Stue  you  come  out  with 
more  knowledge,  but  there  is  no  imderstanding  how  to  use  it. 


Note:  Names  and  inappropriate  comments  were  deleted. 
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